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POREl^DRD 


The  U.  S.  Army  now  has  single  service  responsibility  for  the  pro- 
duction of  conventional  niunitions .    The  majority  of  the  facilities  for 
munition  production  were  acquired  before  and  during  the  World  War  II 
mobilization.    During  the  Southeast  Asian  conflict  it  became  apparent 
that  these  facilities  might  not  be  capable  of  supporting  another  conflict. 
Subsequently,  the  Axniy  initiated  a  $10  billion  program  that  will  run 
through  the  1990 's  to  modernize  the  munitions  production  base.  Addition- 
al bacl<:ground  on  this  program  can  be  found  in  the  Introduction  of  this 
!  report. 


To  effectively  modernize  the  munitions  production  base  requires,  in 
many  cases,  the  use  of  advanced,  conputerized  process  control  systems. 
Process  control  system  technology  has  recently  undergone  rapid  change,  in 
j  part  because  of  increased  uses  of  cctnputers.    To  assist  toward  the  goal 

of  making  proper  and  effective  use  of  this  technology  the  U.S.  Army  in 
1  cooperation  with  the  National  Bureau  of  Standards  has  undertaken  the 
I  development  of  a  Control  Systems  Handbook.    The  Handbook  will  have  a 
I  tutorial  function,  in  that  it  will  provide  a  conrnon  knowledge  base  for 
'  both  military  and  civilian  personnel  vho  are  involved  in  the  specifica- 
tion, design,  procurement,  and  acceptance  of  control  systems.    It  will 
I  also  have  a  reference  function  in  that  it  will  include  annotated  listings 
]  of  all  relevant  standards. 


jj         One  of  the  topics  in  the  Handbook  is  control  systems  readiness.  Be- 
I  cai:ise  of  the  nonuniform  demand  for  munitions,  the  operation  of  these  con- 
l  trol  systons  will  frequently  go  through  a  layaway  phase,  a  standby  phase, 
I  and  a  reactivation  phase.    The  readiness  problem  is  concerned  with  de- 
1  signing  control  systems  to  facilitate  handling  during  these  three  phases. 

i  Since  experiences  in  this  area  are  not  adequately  documented  in  the 

I'  literature,  the  National  Bureau  of  Standards  contracted  with  the  Purdue 
'  Laboratory  for  Applied  Industrial  Control  to  organize  a  workshop  on  con- 
trol systans  readiness.    The  Workshop  brought  together  selected  experts 
.  from  Government,  industry,  and  academic  research  groups.    The  objectives 
of  the  Workshop  were  to  identify  issues  in  the  design  and  operation  of  control 
systens  which  would  facilitate  readiness,  and  reconmend  procedures  to  be 
used  in  the  layaway,  standby,  and  reactivation  phases. 

i!  The  proceedings  of  the  Workshop  consist  of  a  description  of  the 

1  production  readiness  problem,  a  collection  of  twelve  papers  which 

serves  to  identify  the  issues  in  production  readiness,  and  a  set  of 
il  eighteen  consensus  recoimiendations  that  reflect  imny  of  the  issues 
'  and  concerns  expressed  in  the  papers.    We  believe  that  these  proceedings 
will  be  useful  to  Govemirent,  industry,  and  academic  research  groups 
in  improving  their  abilities  to  specify,  design,  and  construct  process 
i]  control  systens  for  production  readiness. 
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ABSTRACT 


Experts  in  the  field  of  Industrial  Process  Control  were  asked  to 
assist  the  U.  S.  Army  and  address  a  most  challenging  and  unique  problem. 
Large  capacity  munitions  manufacturing  production  facilities  are  required 
only  in  times  of  national  emergency.    These  production  facilities  are 
complex  manufacturing  processes  normally  operated  by  sophisticated 
industrial  process  control  systems. 

Long  periods  of  dormant  storage  may  occur  between  facility  construc- 
tion and  required  operation.    Guidance  is  required  for  the  Army  to  prop- 
erly plan  for  appropriate  teclinical  activity  which  will  assure  the 
readiriess  of  these  industrial  processes  at  the  time  of  national  emergency. 
The  Workshop  which  is  reported  upon  in  this  document  was  held  to  help 
provide  some  of  the  needed  background  to  this  problen.    It  also  generated 
a  set  of  recommendations  to  be  considered  by  the  U.S.  Army  vvhen  setting 
up  its  procedures  in  this  area.    In  addition  to  a  set  of  background 
papers,  discussions  were  held  in  the  areas  of  transducers,  magnetic  media 
reliability,  electronic  and  mechanical  element  needs,  testing  and  proving 
of  control  syston  conponents,  reliability  enhancement  techniques,  simu- 
lation for  personnel  training  purposes,  and  the  effect  of  initial  control 
system  design  on  personnel  skill  requirements.    The  recommendations 
developed  treat  each  of  these.    Further  work  is  needed  to  refine  and 
verify  these  reconmendations . 


Keywords:    Readiness,  Layaway,  Standby,  Reactivation,  Autonatic 

Control  Systems,  Dormant  Storage,  Reliability,  Dlagnetic 
Tape,  Simulation,  Sensors 
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EXECUTIVE  SUMMARY 


The  Workshop  on  Control  Systeans  Readiness  for  Munitions  Plants , 
was  held  at  Purdue  University  on  Monday  and  Tuesday,  Septerriber  19-20, 
1977.    The  Vforkshop  was  commissioned  by  the  Office  of  Developmental 
Autcmation  and  Control  Technology  of  the  Institute  for  Computer  Sciences 
and  Technology  of  the  National  Bureau  of  Standards  under  subcontract 
from  the  Itoitions  Production  Base  Modernization  and  Expansion  Program, 
U.  S.  Array. 

The  purpose  of  the  Workshop  was  to  bring  together  a  group  of  those 
government  personnel  ^o  are  concerned  with  the  planning  for  and  the  de- 
signing of  the  control  systems  involved  in  the  modernization  and  expan- 
sion of  our  nation's  munitions  plants,  along  with  a  group  of  technical 
experts  in  computer  control  system  development  and  installation  in  the 
plants  of  civilian  industry.    The  two  groups  then  exchanged  information 
concerning  the  U.  S.  Amty-'s  plans  for  the  modernization  and  expansion 
program.    The  difficulties  xiiich  the  military  foresaw  in  installing 
modem  control  systems  in  these  plants  and  then  deactivating  them  for  a 
period  of  years  before  eventually  requiring  production  were  developed. 
The  technical  experts  then  responded  to  present  their  concepts  of 
the  design,  installation,  and  storage  precautions  v^ich  should  be  taken, 
the  procedures  for  maintenance  and  readiness  testing  to  be  used  during 
the  inactive  period,  and  the  requirements  to  be  established  for  the  re- 
activation itself. 

A  large  part  of  the  discussions  of  the  WDrkshop  were  directed  by 
the  set  of  twelve  papers  T^hich  are  included  in  this  report.    The  first 
three  papers  described  the  Production  Base  Ifodemization  and  Expansion 
Program,  the  difficulties  with  previous  munitions  plants  vMch  prompted 
it,  and  the  problems  and  difficulties  Ti^ich  were  foreseen  by  the  military 
planners  and  designers . 

The  next  set  of  five  papers  described  the  requirement  for  sensors 
in  terms  of  the  layaway  problon,  precautions  and  capabilities  concerning 
magnetic  tape  files  of  computer  programs  and  plant  operating  data,  the 
environmental  effects  on  electrical  and  electronic  equipment  under  long 
storage,  handling  of  valves  and  other  mechanical  elements,  and  methods 
for  testing  and  proving  out  conputer  hardware  and  software  elements. 

The  operation  of  a  beet  sugar  mill  bears  sane  resenblance  to  a 
munitions  plant  in  that  it  operates  periodically.    A  munitions  plant  is 
different  because  the  frequency  and  duration  of  the  period  is  not  known, 
and  because  it  requires  more  advanced  control  systems.  Nonetheless, 
this  industry's  experiences  are  relevant,  and  therefore  a  paper  was 
presented  describing  the  maintenance  activities  during  the  period  be- 
tween active  campaigns,  and  the  staffing  practices  and  problens  engen- 
dered by  the  periodic  operation. 
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The  last  three  papers  described  the  applicability  of  modem  high 
reliability  techniques  vMch  should  soon  be  available  with  the  reduction 
in  price  of  conputer  hardware,  the  use  of  simulation  as  a  training  aid 
for  bringing  new  crews  rapidly  up  to  speed  during  the  reactivation 
period,  and  the  effect  T;^Mch  modem  plant  and  control  system  design 
techniq^oes  can  have  in  reducing  the  need  for  highly  skilled  operating 
personnel.    The  reliability  paper  emphasized  the  trends  in  new  conputer 
control  systsiis  made  possible  with  integrated  circuit  elements  and  the 
new  reliability  techniques.    The  simulation  ^d  training  paper  described 
operator  training  work  that  uses  the  techniques  of  ccnputer  simulation 
to  present  realistic  plant  type  problems  to  these  personnel  vMle  in 
training.    The  final  paper  presented  sane  of  the  capabilities  of  modem 
systans  engineering  techniques  to  ease  operator  tasks,  particularly  in 
the  interpretation  of  plant  data  in  emergency  situations,  and  thus  lower 
the  training  requirements  for  these  personnel. 

The  major  results  of  the  I^rkshop  are  presented  in  the  "Recarmenda- 
tions"  section.    These  results  are  a  set  of  eighteen  consensus  recotimen- 
dations  that  were  reached  by  the  participants  during  discussions  that 
took  place  the  second  day  of  the  Vforkshop.    The  reccomendations  concem 
issues  and  procedures  for  control  system  readiness.    They  reflect  many 
of  the  specific  concerns  discussed  in  the  individual  papers.    They  are 
grouped  into  the  following  topic  areas:    General  and  Organizational, 
Conputer  Hardware,  Conputer  Software,  Other  Systems  Considerations  (e.g. 
Instrumentation,  ^lechanical  Elements),  and  PersOTinel. 

Appendix  I  presents  a  list  of  all  participants.    Appendix  II  shows 
the  agenda  and  program  of  the  Workshop. 
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INTRDDUCriON 


EACKGRQUND 

The  conventional  mLinitions  used  by  United  States  forces  and 
si:5)ported  allies  are  manufactured  in  a  network  of  government -owned, 
government  operated  (GOGO)  and  government-owned,  contractor  operated 
(COCO)  arnnunition  plants  supported  by  a  vast  network  of  contract  or- owned, 
contractor  operated  (CDCD)  facilities.    Typically  the  government  owned 
facilities  (GOGO,  GOGO)  are  involved  in  the  production  of  lethal  pro- 
pellants  and  explosives  and  the  loading,  assembly  and  pack-out  of  lethal 
nmitions.    This  is  done  to  retain  a  base  that  would  not  otherwise  be 
available  in  industry.    On  the  other  hand,  industry  owned  facilities 
sijpply  a  n^iad  of  coirponents  and  end  items  that  include  cartridge 
cases,  fuzes  and  shell  metal  parts.    Items  manufactured  by  the  base  in- 
clude a  wide  array  of  sizes  and  shapes  of  ammunition  ranging  frcm  5 . 56mm 
small  arms  cartridges  costing  six  cents  to  2,000  pound  boiibs  costing 
$17,000  each.     (A  typical  artillery  round  is  shown  in  Figure  1).  The 
plants  included  in  the  production  base  contain  over  250  production  lines. 

These  lines  may  be  classified  according  to  the  production  function 
they  perform.    Propellant  and  explosive  (P&E)  facilities  include  lines 
to  produce  all  types  of  propellants,  high  explosives  and  p3T:otechnic 
compositions  as  well  as  explosives  ingredients  such  as  nitrocellulose, 
nitroglycerin  and  acids.    Ifetal  parts  C'^S)  lines  manufacture  shell 
and  projectile  bodies  for  mortar,  tank  and  artiillery  rounds.    They  also 
manufacture  bombs,  cartridge  cases,  fuzes  and  small  metal  parts.  The 
propellants  and  explosives,  metal  parts  and  conponents  for  munitions 
items  are  loaded,  assembled  and  packed  in  what  are  known  as  LAP  facili- 
ties.   These  facilities  usually  include  equipment  for  the  melting  and 
pouring  of  high  explosive  into  the  munition  along  with  finishing, 
assenbly  and  packout  operations.    Small  arms  lines  combine  the  generic 
functions  of  metal  parts  and  LAP  facilities  into  one  ccmplex  due  to  their 
small  size  and  extranely  high  production  rates. 

The  munitions  production  base  is  distributied  into  28  Army  plants, 
and  131  ooimercial  sites  as  depicted  in  Figures  2  and  3.    This  munitions 
production  base  is  managed  by  the  US  Army  Armament  Materiel  Readiness 
Command  (ARRCCM) ,  headquartered  in  Rock  Island,  IL.    ARRCQM  is  responsible 
for  procurement,  production  and  logistics  management  of  conventional 
airmunition  for  the  Navy  and  Air  Force  as  well  as  the  Army.    ARRCCM  work- 
loads the  base  during  peacetime.    When  there  is  no  work  for  a  particular 
plant  then  that  plant  is  shut  down  and  laidaway  for  future  use.  ARRCQM 
then  has  the  responsibility  for  maintaining  the  facilities  and  equipment 
in  a  state  ready  for  rapid  reactivation  in  case  of  a  surge  in  requirements 
or  general  mobilization.    The  present  goal  is  to  be  able  to  start  production 
within  90  days  of  initiating  reactivation.    To  help  achieve  this  goal,  ARRCOM, 
in  conjunction  with  EARCOM's  Industrial  Base  Engineering  Activity  and  the 
Project  Manager,  MPBME,  is  looking  at  process  control  systems  as  a  potential 
aid  in  reducing  reactivation  time. 
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As  previously  mentioned,  the  great  majority  of  metal  parts  required 
for  munitions  manufacture  are  produced  by  the  private  sector.  However, 
there  is  no  industrial  base  for  many  of  the  largest  munitions  items, 
and/or  in  some  cases  insufficient  capacity  to  satisfy  overall  needs.  In 
these  instances,  government- owned,  contractor-operated  plants  fill  the 
void. 

Small  caliber  amnLmition  for  military  rifles  and  carbines  is  very 
similar  to  rounds  for  sporting  rifles  and  the  production  lines  are  almost 
identical.    However,  the  capability  of  industry  is  limited  so  that  the 
full  requirements  for  military  rounds  cannot  be  acconnodated.  Accordingly, 
government-owned,  contractor-operated  production  plants  are  required  to 
satisfy  mobilization  reauironents . 

In  peacetime,  munitions  requirements  are  extremely  small  as  com- 
pared to  full  mobilization  requirements.    As  a  consequence  of  this  non- 
uniform production  requirement,  nearly  the  entire  initial  amnunition 
consunption  is  satisfied  from  inventory  stockpiles  during  mobilization. 
As  the  production  base  comes  up  to  speed,  more  of  the  amnunition  con- 
sumption will  be  satisfied  directly  from  production  thereby  lowering 
the  stockpile  drain  (see  Figures  4  and  5).    Clearly,  decreasing  the  re- 
sponse time  of  the  production  base  reduces  the  inventory  stockpile  re- 
quirement.    Studies  indicate  a  cost  avoidance  of  approximately  one 
billion  dollars  for  each  month  that  response  time  can  be  reduced.  This 
figure  does  not  include  the  cost  of  transportation,  storage,  maintenance 
or  obsolescence  of  munitions  in  the  stockpile. 

The  majority  of  the  production  base  was  acquired  or  constructed 
during  the  World  War  II  mobilization.    These  obsolete  and  in  seme 
instances  deteriorated  facilities  produced  amnunition  for  three  wars 
after  long  periods  of  inactivity.    At  the  peak  of  the  Southeast  Asian 
conflict,  it  became  apparent  that  the  production  base  might  not  be 
ready  for  the  next  conflict.    First,  the  equipment  was  WDm  out.    It  was 
very  labor  intensive,  had  extensive  safety  and  health  problems  and  was, 
in  some  cases,  a  very  serious  polluter.    In  addition,  it  could  not  manu- 
facture a  host  of  new  generation  munitions  items  that  were  being 
developed.    This,  along  with  a  relatively  slow  mobilization  response 
persuaded  defense  officials  of  the  need  to  ccoipletely  overhaul  and 
modernize  the  amnunition  plants.  In  1970,  the  Army  launched  a  compre- 
hensive program  of  unprecedented  scope  to  update  the  amnunition  produc- 
tion base  from  a  circa  1930  vintage  into  a  system  that  would  be  respon- 
sive to  the  needs  of  the  1980 's  and  1990' s. 

PRODUCTION  BASE  MDDERNIZATION  PROGRAM 

In  recognition  of  the  criticality  of  the  efforts  to  provide  the 
nation  with  a  modernized  amnunition  production  base  of  the  proper  size, 
the  Secretary  of  the  Army  established  a  Project  Manager  (PM.)  for  the 
manageanent  of  the  >bnitions  Production  Base  I-lodemi-zation  and  Expansion 


Program.    The  PM  exercises  centralized  management  authority  over  the 
planning,  direction,  control  and  execution  of  the  production  base 
mDdemization  and  ejq^ansion  program  at  all  U.  S.  Army  amnunition  plants 
and  arsenals  and  for  the  government  equipment  located  at  contractor- 
owned  and  operated  facilities  included  in  the  program.    The  RI  office 
is  located  at  the  Armament  Research  and  Development  Comnand  (ARRADCCM) , 
Dover,  N.J. 

This  diverse  and  conplex  program  is  now  in  its  eighth  year  and  will 
be  completed  in  the  1990' s.    At  a  cost  of  $10  billion  (excluding  infla- 
tionary growth),  it  is  one  of  the  largest  facilities  programs  ever  under- 
taken in  peacetime  by  either  the  government  or  private  industry.  As 
currently  structured,  the  program  is  made  up  of  about  600  individual 

I  projects  for  facilities  and  about  400  supporting  manufacturing  engineer- 

j  ing  projects. 

I 

The  size,  scope,  diversity  and  ccmplexity  of  the  program  are  awe- 
1  some.    But  with  the  establishment  of  a  M,  the  utilization  of  the  col- 
i  lective  expertise  and  experience  of  the  participating  organizations, 
j  and  the  development  of  new  planning,  programmng  and  execution  methodo- 
logies, the  mission  is  proceeding  in  an  orderly  manner  toward  its  goal 
of  a  fully  responsive  ammunition  production  base  at  an  affordable  cost. 

Modernization  is  a  broad  term  denoting  the  inprovement  of  indus- 
trial facilities  through  replacement,  modification,  rearranganent ,  or 
addition  of  capability  in  order  to  achieve  economic,  quality,  time  or 
safety  advantages.    Ifrider  this  program,  a  partially  or  completely  new 
system- engineered  production  line  or  process,  incorporating  the  latest 
proven  manufacturing  technology,  replaces  an  obsolete  or  non-economic 
facility.    In  essence,  this  will  inprove  readiness,  abate  pollution  and 
correct  OSHA  deficiencies.    Based  upon  DoD  directives,  the  defense 
establishment  has  been  charged  to  set  the  exanple  for  industry  in  the 
fields  of  pollution  abatement  and  safe  working  conditions  (OSHA) .  How- 
ever modernization  does  not  supplant  restoration  and  replacement  projects 
for  current  production  support,  although  it  might  incidentally  achieve 
such  a  goal.    The  addition  of  new  or  inproved  facilities  is  classified 
as  an  expansion  effort  when  justified  by  expanding  current  procurement 
or  increased  mobilization  requirements.    An  expansion  of  an  annunition 
production  facility  usually  incorporates  the  latest  proven  manufacturing 
technology;  therefore,  modernization  is  also  associated  with  expansion. 

As  previously  noted,  the  munition  production  base  prior  to  1970  in- 
corporated technologies  of  the  1930' s.    The  key  aspects  of  this  program 
are  to  reduce  the  use  of  critical  materials,  energy  and  skills  through 
the  use  of  inproved  manufacturing  technology. 

In  addition  to  the  economic  advantages  of  more  efficient  production 
facilities  producing  lower  unit-cost  items,  a  modernized  production  base 
will  result  in  improved  responsiveness  of  the  base.    The  new  production 
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base  will  shorten  the  time  reqiiired  to  reach  a  production  rate  eqijal  to 
consumption  rate;  these  factors  decrease  the  quantities  of  amnunition 
^^ch  must  be  stockpiled  to  support  inobilization  reqioirements . 

MANUFACTURING  TECHNOLOGY  PROGRM 

Along  with  every  other  consumer,  the  Army  has  been  hit  hard  in  the 
pocketbook  by  a  steeply  inflationary  econctny.    The  effects  of  inflation 
must  be  counteracted  by  increased  productivity^     Historically,  increased 
productivity  has  been  achieved  by  installing  processes  incorporating  the 
latest  and  best  technology.    This  briefly  is  the  rationale  vhich  justi- 
fies the  DoD  manufacturing  technology  program.    Some  of  the  results 
achieved  in  this  program  have  resulted  in  major  inprovements  in  manu- 
facturing methods  in  the  civilian  sector  in  addition  to  their  contribu- 
tion in  the  production  of  defense  items.    Modernization  of  the  Army's 
outmoded  production  facilities  is  well  underway  particularly  at  govern- 
ment-owned P&E  and  LAP  facilities .    The  next  area  emphasized  will  be 
MPTS  both  in  government  owned  and  the  industrial  owned  sectors.    Seme  of 
the  new  technologies  are  ready  for  production  adaptation  now;  others 
will  be  ready  in  the  next  several  years. 

In  the  P&E  area,  modernization  projects  have  been  undertaken  to 
eliminate  the  air  and  water  pollution  associated  with  the  older  processes. 
New  acid  production  facilities  have  been  constructed  incorporating  the 
latest  industrial  technology.    Future  year  effort  has  been  allocated  to 
the  construction  of  single  and  multi-base  propellant  plants. 

The  MPTS  modernization  program  began  in  the  early  1970 '  s  with  the 
updating  of  forging  and  heat  treating  facilities  at  one  of  the  GOCO 
plants .    Equipment  being  installed  in  the  MPTS  program  will  be  capable 
of  processing  not  only  current  amnunition  designs  in  carbon  steels  but 
all  the  new  generation  munitions  that  will  require  alloy  steels  and 
high- fragmentation  alloys.    The  small  caliber  amnunition  modernization 
program,  known  as  SCAMP,  is  currently  conpleting  its  prove-out  phase  and 
will  ultimately  produce  amnunition  at  rates  previously  deemed  unattain- 
able. 

The  LAP  modernization  program  is  divided  into  three  broad  categories; 
melt-pour  facilities,  projectile  and  component  facilities,  and  support 
facilities.    The  major  thrust  is  in  the  melt-pour  area.    The  new  process 
of  continuous  melt-pouring  of  TNT,  Caiposition  B,  or  other  high  explo- 
sives into  artillery  shells  has  been  piloted  at  ARRADCOM. 

Due  to  the  diminishing  horological  industry  in  the  United  States 
since  the  1930' s,  the  DoD  has  relied  heavily  on  imported  precision  fuze 
conponents  to  support  its  mobilization  posture.    Subsequent  to  the 
Korean  War,  an  intensive  effort  has  been  made  by  the  Army  to  modify  fuze 
designs  and  exploit  new  fuzing  concepts  to  minimize  dependence  on  foreign 
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sources  of  oonponents.    The  inodemization  program,  in  addition  to  auto- 
mting  the  fuze  production  base,  incorporates  new  designs  which  eliminate 
the  government's  forrrer  predominant  reliance  on  non~ indigenous,  preci- 
sion-type watch  producers. 

The  application  of  modem  processes  and  manufacturing  technology  to 
the  munitions  production  task  requires,  in  many  cases,  the  use  of  modem 
process  control  systems.    Further,  it  is  expected  that  some  present  and 
future  control  systems  will  provide  capabilities  x^diich  will  greatly  en- 
hance the  readiness  posture  of  modem  production  facilities.    The  poten- 
tial increased  benefits  available  from  modem  control  systems  technolo- 
gies are,  however,  accompanied  by  some  new  problems.    Little,  if  any, 
experience  base  exists  conceming  the  long  term  (5  to  10  years)  dormant 
storage  of  classical  and  modem  (analog/digital)  industrial  electronic, 
optic,  fluidic,  and  pneumatic  conponents  and  systems.    The  definition 
of  procedures  for  layaway,  standby  and  reactivation  of  control  systems 
is  required  to  assure  that  the  best  possible  approach  is  chosen  to 
accurately  predict  the  readiness  posture  of  the  munitions  production 
base  using  these  systans.    Further,  the  definition  of  control  system 
design  standards  is  expected  to  enhance  design  uniformity  and  familiarity 
wit±i  system  documentation  (design ,  operations ,  maintenance ,  etc . )  . 

READINESS  PROBLEM 

The  production  facilities  management  problem  can  be  grossly  segre- 
gated into  two  main  categories.    The  first  area  is  the  system  or  ccm- 
ponent  degradation  in  the  dormant  (nonproduction)  environment.  The 
second  area  is  the  human  element  of  the  process  and  the  learning  curve 
^ich  must  be  traversed  at  mobilization  to  effect  full ,  reliable  pro- 
duction.   The  Army  is  in  need  of  technical  guidelines  to  address  both 
these  areas.    The  ARPCQM  participants  specifically  were  searching  for 
solutions  to  the  problan  of  layaway,  maintenance  and  reactivation  of 
process  control  systems  over  long  periods  of  time  (10-15  years)  including 
continual  vendor  support  and  equipment  obsolescence.    Before  process 
control  systems  can  be  adopted  as  a  solution  to  reducing  reactivation 
time,  an  assessment  must  be  made  regarding  the  practicability  of  laying 
away  and  reactivating  these  same  systems,    ^1aintenance  procedures  must  be 
developed  \A^ich  will  assure  rapid  reactivation.    Therefore  the  goal  of 
this  workshop  was  to  address  this  practicability  and  subsequently  to 
develop  guidelines  governing  the  layaway,  maintenance  and  reactivation 
of  process  control  systens.    These  general  guidelines  will  be  used  to 
generate  design  and  logistics  guidelines  (standards)  and  procedures  to 
be  used  as  a  matter  of  policy  for  modernized  munitions  production  systeiis. 
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Technical  position  papers  were  solicited  from  a  large  group  of 
such  experts.    These  papers  were  then  distributed  to  all  other  participants 
prior  to  the  ineeting.    All  participants  reviewed  the  papers  and  were  pre- 
pared to  discuss  each  paper  in  perspective  with  the  other  papers  at  the 
Workshop.    The  individual  papers  and  discussions  will  be  used  by  the 
government  to  formulate  and  inplement  a  set  of  procedures  to  assist  in 
reaching  the  goal  of  a  suitable  readiness  program. 
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RECOMMENDATIONS 


The  first  day  and  one  half  of  the  Vforkshop  was  spent  presenting 
the  contents  of  the  technical  papers.    As  each  paper  was  presented, 
participants  actively  expressed  their  thoughts  regarding  the  significance 
of  its  contents  to  the  control  systems  readiness  problem.    These  presen- 
tations and  discussions  formed  a  basis  for  the  afternoon  of  the  second 
day,  vMch  was  spent  formulating  the  following  eighteen  recCTimendations . 

The  list  of  Vforkshop  participants  contained  in  Appendix  I  shows 
that  the  assembled  group  represented  a  widespread  knowledge  base  and  set 
of  experiences .    Representatives  from  industry  (both  users  and  suppliers 
of  control  systems),  Government,  and  academic  research  groips  with  ex- 
periences in  both  specialized  aspects  as  well  as  system  responsibilities 
were  present.    Such  a  diverse  group  provided  a  broad  yet  technically 
deep  basis  for  the  recomnendations . 

The  recanmendations  are  grouped  into  five  topic  areas.    The  first 
group  is  concerned  with  general  techniques,  procedures,  and  require- 
ments vdiich  are  applicable  at  a  systore  level.    These  reccmnendations 
are  largely  influenced  by  industry  practices  as  related  by  the  indus- 
trial participants.    The  second  and  third  groups  concern  practices  re- 
lated to  the  use  of  ccanputers  to  irtplement  the  control  aspect  of  a 
system.    There  are  reoomnendations  for  hardware  and  software.  The 
fourth  group  includes  recatmendations  related  to  sensors,  mechanical 
elements,  paver  supply,  and  communication  elements  of  a  system.  The 
final  group  of  recommendations  speaks  to  personnel  aspects  of  control 
systens  readiness. 

The  recommendations  themselves  are  based,  of  course,  on  the  aggre- 
gate of  the  experiences  of  the  participants.    Many  of  then  have  proven 
both  practical  and  beneficial  in  various  industrial  applications.  It 
was  agreed  that  the  implementation  of  the  ccmplete  set  of  reccmmenda- 
tions  is  ambitious,  but  is  required  to  meet  the  Army's  readiness  goals. 
The  reconmendations  were  formulated  by  all  participants,  and  thus  re- 
present a  consensus  agreement.    There  was,  however,  a  diversity  of 
opinions  expressed  regarding  such  things  as  the  relative  importance  of 
individ-ual  reconmendations  and  mechanisms  for  implementing  individual 
reconmendations.    The  unanimous  agreement  on  the  anibitioijsness  of  the 
task,  and  the  diversity  of  the  opinions  expressed  underscores  the 
difficulty  of  inplementation. 

The  underlined  text  constitutes  the  recomnendations  as  formulated 
at  the  WDrkshop.    The  explanation  vMch  follows  each  recoimendation 
gives  relevant  background  discussion.    To  varying  degrees  these  explana- 
tions answer  the  v^t,  the  ^y,  and  the  how  of  each  reccranendation. 
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GENERAL  AND  ORGAI^IZATIONAL 

1.  Systems  should  be  designed  so  that  all  hardware  and  software 
features  are  organized  on  a  well-recognized  modular  basis . 

Every  industrial  plant  is  different  from  its  fellows  in  ways 
which  nearly  always  prevent  the  direct  transfer  of  conplete  control 
systems  designs  from  one  to  the  other.    At  the  same  time,  all 
plants  are  constructed  of  different  arrangements  and  combinations 
of  very  similar  building  blocks  or  modules.    These  modules  are  based 
on  functional  requirements  for  systems  sx.ich  as  those  identified  by 
the  International  Purdue  Workshop  on  Industrial  Conputer  Systems 
(Significant  Accomplishments  and  Documentation  of  the  Intemational 
Purdue  Workshop  on  Industrial  Conputer  Systems ,  Part  IV,  pp.  11-57). 
Designing  modular  hardware  and  software  systems  necessitates  the 
use  of  standardized  ccramunications  links,  structural  software 
engineering  techniques,  and  modular  operator  interface  components. 
Designing  on  a  nodular  basis  minimizes  the  engineering  effort  that 
goes  into  a  new  plant,  facilitates  the  maintenance  of  existing 
plants,  and  simplifies  the  difficulties  of  responding  to  unexpected 
demands. 

2.  Each  system  should  be  inplemented  and  handled  during  all  phases  by 
a  user  team  \hlch.  includes  government  and  plant  operating  person- 
neT.    Those  disciplines  lacking  at  "EEe  plant  location  should  be 
supplied  by  an  in-house  central  control  systems  gro\ip.    The  central 
control  systegns  group  should  have  an  approval  function  in  order  to~ 
preserve  Army  obj  ectives  and  policies . 

The  Array  must  insure  that  it  is  involved  in  functional  speci- 
fication, design,  inplementation,  prove  out,  and  indeed  in  all 
intermediate  aspects  of  the  construction  of  a  plant.    The  involve- 
ment must  include  personnel  from  a  central  control  systems  support 
group,  as  well  as  personnel  vho  are  concerned  with  the  day  to  day 
operation  of  the  plant.    The  involvement  of  the  support  group  is 
necessary  so  that  a  high  level  of  technical  competence  and  the 
broad  perspective  of  a  central  group  can  be  maintained.    Only  a 
central  support  group  can  enforce  a  basic  conmonality  among  sys- 
tems at  different  sites,  which  often  are  designed  and  constructed 
by  different  companies  (a  mechanism  for  this  enforcement  is  des- 
cribed in  Recomnendation  3) .    The  involvement  of  the  plant  opera- 
ting personnel  is  necessary  so  that  basic  knowledge  of  the  system, 
as  well  as  identification  with  the  system  can  be  established  by 
the  people  who  will  some  day  run  the  plant.    This  knowledge  and 
identification  will  avoid  a  reversion  to  manual  operation  should 
system  difficulties  occur,  and  will  facilitate  maintenance  and 
repair.    A  conplete  dependence  on  external  personnel  at  such  times 
can  involve  unacceptable  delay  and  expense. 
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The  tiNO  major  user  corporations  represented  at  the  Vforkshop 
have  central  support  groups,  and  involve  these  groups  and  plant 
operating  personnel  in  all  phases  of  plant  implementation.  The 
corporations  are  large  users  of  advanced  control  systems,  and 
consider  these  policies  absolutely  essential  to  meet  the  corporate 
goals  of  effective  use  of  advanced  control  systems. 

The  government  should  develop  a  set  of  system  performance,  appli- 
cation design,  evaluation,  documentation,  and  procurgnent  standards 
vfeich  will  be  the  basis  for  the  carrying  out  of  the  recomnendations 
stated~H^ein. 

A  primary  mechanism  for  implementing  the  Vforkshop  recamienda- 
tions  is  the  development  of  the  above  standards.    The  central 
support  group  (described  in  Recommendation  2) ,  xvould  use  the 
standards  as  a  major  tool  to  acconplish  its  task.    The  standards 
must  be  developed  to  satisfy  the  Army's  control  systons  requirements 
for  munitions  plants.    The  standards  should  enccrapass  all  aspects 
of  a  project  life  cycle,  including  procedures  for  functional  re- 
quirements specification,  design,  performance,  documentation,  and 
acceptance  testing. 

Participating  representatives  of  corporations  that  are  users 
of  control  systems  reported  that  they  use  such  internal  standards 
and  procedures.    Thus,  as  is  the  case  for  Recoimendation  2, 
accepted  and  proven  practices  in  industry  are  the  basis  for  this 
recCTimendation . 

Systgns  to  be  procured  should  have  an  established  application 
history  and  consideration  must  be  given  to  the  future  programning 
and  maintenance  support  availability  of  similar  systems .    A  ten 
year  minimum  period  is  considered  desirable . 

Industry  experience  has  shown  the  undesirability  of  the  use  of 
a  new  computer  design  in  a  plant  control  system  application.  This 
is  because  such  systems  tend  to  have  many  "bugs"  or  design  and 
programming  errors  vMch  have  not  yet  been  uncovered  throi:igh  con- 
tinued testing  and  use.    It  has  been  shown  that  a  plant  installa- 
tion period  of  about  two  years  is  necessary  to  assure  that  a  new 
conputer  system  design  has  received  adequate  testing  and  use  to 
assure  its  future  viability. 

An  established  application  history  also  helps  to  answer  the 
question  as  to  v^ether  the  conputer  system  vendor  intends  to  sup- 
port this  particular  conputer  system  with  spare  parts  and  program- 
ming aids  for  the  life  of  the  plant  system  \vhich  has  just  been 
installed.    A  minimum  life  of  at  least  ten  years  is  forecast  for 
most  industrial  plant  control  systems  today.    Spare  parts  and 
programning  aids  must  be  available  from  the  vendor  for  at  least 
this  period  of  time. 
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This  recomneindation  can  be  inplemented  by  incorporating  it  in- 
to procurement  standards  and  procedures. 

5.  |fo  control  system  enhancements  should  be  made  other  than  those 
necessary  to  maintain  currently  reciuired  performance  capability 
vMle  in  the  dormant  phase.    Equipment  should  be  replaced  only 
\jhen.  oHsolete  and  no  longer  supported  by  the  vendor. 

The  rate  of  development  in  computer  control  systems  is  so 
rapid  today  that  most  systems  are  obsolete  almost  as  soon  as  they 
are  installed  -  they  are  still  perfectly  capable  of  performing 
their  assigned  tasks  but  are  nevertheless  no  longer  up  to  the 
state-of-the-art.    The  natural  desire  to  have  an  up  to  date  system 
is  a  strong  incentive  to  make  alterations  or  enhancements  to  exist- 
ing and  otherwise  capable  systems.    These  enhancements  are  not  only 
expensive  in  manpower  needs,  costs,  and  plant  downtime,  but  will 
often  cause  many  other  problems  in  the  system  if  the  interaction  of 
the  new  functions  with  existing  ones  are  not  fully  mderstood. 
Therefore  such  enhancements  for  the  sake  of  modernization  alone 
should  be  strenuously  avoided.    Likewise  the  replacanent  of  an 
existing  satisfactory  systen  with  a  newer,  less  expensive,  more 
capable,  etc.,  system  should  also  be  strenuously  avoided,  unless 
the  present  system  is  no  longer  supported  by  the  vendor  for  spare 
parts  and  progranming  aids. 

This  recomnendation  can  be  put  into  practice  by  establishing 
an  enhanconent  and  replacement  policy. 

6.  Simulation  is  an  excellent  method  of  verifying  the  design  of 
equipment  and  controls  in  critical  situations.    It  should  be  used 
v^enever  such  verification  is  desirable  and  feasTEle. 

Simulation  is  the  di^Dlication  of  the  behavior  of  a  given 
system  by  another  system  or  by  a  conputer.    The  duplication  of  be- 
havior is  acconplished  by  developing  a  model  of  the  given  system 
based  on  mathematical,  chemical,  and  engineering  analyses.  Increas- 
ingly, the  computer  is  being  used  as  an  effective  tool  to  inplement 
the  model. 

The  advantages  of  simulation  center  around  being  able  to  de- 
rive information  about  the  given  system  prior  to  its  actual  con- 
struction.   Simulation  allows  for  a  deep  understanding  of  the 
processes  \(^ich  are  part  of  the  systen;  it  makes  it  possible  to 
perturb  systen  parameters  and  thus  evaluate  alternative  designs 
and  devices.    Still  another  advantage  of  simulation  (described  in 
Recommendation  18)  is  that  it  can  be  the  basis  of  an  ongoing 
training  program. 

A  potential  disadvantage  of  simulation  is  its  dependence  on 
models.    The  evaluations  derived  from  the  simulation  are  valid 
only  to  the  extent  that  the  model  used  actually  represents  the 
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processes  that  are  part  of  the  syston.    A  poor  model  will  yield 
questionable  evaluation.    However,  this  should  not  be  interpreted 
too   negatively,  since  the  state  of  the  art  of  model  building  is 
good,  and  it  is  possible  (based  on  considerations  such  as  past 
experiences  with  similar  processes)  to  verify  models.  Sinolation 
requires  manpower  to  carry  out  the  necessary  studies  and  equipment 
to  perform  the  required  ccmputations .    Inadequate  planning  can 
result  in  excessive  cost  for  these  resources. 

Simulation  has  high  potential  benefit  in  the  areas  of  process 
understanding,  alternative  design  evaluation,  and  training.  A 
realization  of  these  benefits  requires  a  ccmnitment  to  the  techni- 
que itself,  as  well  as  a  comiLtment  to  building  up  an  inhouse 
group  of  technical  expertise  that  is  part  of  the  central  support 
group  described  in  Reccoinendation  2. 


The  reccramendations  contained  in  the  following  three  sections, 
COMPUTER  HARDl^^RE,  CCMPUTER  SOFimRE,  and  OTHEH  SYSTEM  CONSIDERA- 
TIONS, can  be  implemented  by  incorporating  them  into  developed 
procurement  procedures  and  readiness  policy. 

CCMPLTTER  HARDWARE 

7.  The  reccannended  layaway  procedure  for  a  cctnputer  system  is  in  the 
power-off  state  in  a  humidity  and  temperature  controlled  area 
with  a  periodic  recycling  to  insure  operabilityT    Recycling  is  to 
be  done  by  the  vendor  or  aser  at  the  option  of  the  user. 

There  has  been  considerable  controversy  in  recent  years  as  to 
the  best  method  of  handling  electronic  equipment  v^le  not  in  use. 
Should  it  be  turned  off  or  should  power  be  kept  on  even  when  not 
in  use.    Recently  it  has  been  determined  that  the  power-off  state 
is  probably  acceptable  for  long  dormant  periods.    At  the  same  time, 
the  necessity  for  a  humidity,  tenperature,  and  environmentally 
controlled  storage  space  has  been  reaffirmed  because  of  the  sensi- 
tivity of  electronic  equipment  to  variations  in  these  conditions. 

A  periodic  activation  of  the  equipment,  probably  at  yearly 
intervals,  is  necessary  to  assure  that  it  will  be  operable  should 
reactivation  of  the  plant  beccme  necessary  in  a  national  emergency. 
This  periodic  activation  and  test  is  an  excellent  training  device. 
Therefore  user  personnel  should  make  such,  tests  if  they  are  avail- 
able for  the  task. 

8.  At  recycling  time  and  at  reactivation  the  system  should  be  ccmpletely 
restored  to  a  new  machine  operational  condition.    The  original  pro- 
gram and  hardware  syston  checkout  should  be  repeated. 

The  only  way  to  assure  that  a  computer  system  reactivated  after 
a  long  dormant  period  is  caipletely  operational  is  to  subject  it  to 
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the  same  tests  \^ch  are  used  to  prove  out  a  new  machine.  There- 
fore these  same  tests  should  be  used  at  each  activation  test  (one 
year  intervals)  of  the  syston  and  at  plant  reactivation  itself. 

9.      Complete  wiring  diagrams,  circuit  board,  and  chip  specifications 

and  designs ,  etc . ,  should  be  available  for  all  ccmputer  and  related 
electronic  hardware. 

A  defense  establishment  such  as  a  munitions  plant  must  not 
allow  itself  to  be  dependent  upon  any  one  individual  or  any  one 
conpany  for  its  future  maintenance  and/ or  spare  parts.  Therefore 
it  is  necessary  that  a  ccoplete  set  of  wiring  diagrams,  circuit 
board  layouts,  chip' specifications  and  designs,  etc.,  used  in  the 
ccffiputer  or  other  electronic  equipraent  be  supplied  with  then.  If 
the  primary  vendor  should  fail,  this  hardware  documentation  allows 
for  the  developtnent  of  alternative  sources  of  maintenance  and 
spare  parts. 


COMPUTER  SOFTWARE 

10.      Proj ect  generated  program  system  source  code  should  be  written  in 
a  high-level,  real-time  language  such  as  FORTRAN  which  is  as 
close  as  possible  to  existing  standards . 

Recent  work  in  the  comparison  of  program  developanent  costs 
has  shown  the  su^jeriority  of  high  level  languages  (over  assembly 
languages)  in  reducing  the  overall  costs  of  conplex  computer  pro- 
grams, such  as  for  computer-based  control  systems.    The  use  of 
standard  versions  of  a  language  greatly  enhances  the  transport- 
ability of  software.    This  will  encourage  the  software  modular 
construction  techniques  contained  in  Recoimendation  1. 

There  has  been  considerable  work  recently  to  develop  standards 
for  process  control  versions  of  the  Fortran  language.  Since 
Fortran  is  the  most  comrionly  used  and  the  most  highly  standardized 
of  the  high  level  languages  for  real  time  control,  it  should  be 
the  number  one  candidate.    WDrk  toward  standardizing  other  more 
advanced  procedural  languages  is  beginning,  and  these  efforts 
should  be  monitored  by  the  central  support  groi:^. 

Sane  participants  pointed  out  the  advantages  of  fill-in-the- 
forms  languages.    With  these  languages  a  user  specifies  vihat  he 
wants  the  computer  to  do  by  entering  the  appropriate  information 
into  a  form  or  table.    The  advantages  of  these  languages  are  the 
ease  of  prog^ramiiing  and  their  self-documenting  nature.    Often  an 
applications  engineer,  with  only  a  minimal  knowledge  of  the  com- 
puter system,  can  fill  in  the  forms.    The  documentation  of  the 
program  is  inherent  in  the  forms.    The  disadvantages  of  fill-in- 
the- forms  languages  is  their  rigid  structure  (conpared  to  FORTRAN)  , 
and  the  fact  that  each  vendor  has  his  own  language.    To  a  certain 
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extent  the  use  of  functional  criteria  (see  Reccffimendation  1)  in 
control  systans  design  alleviates  the  vendor- dependence  disadvan- 
tage. 

11.  All  vendor  generated  programs  and  operating  systems  must  be 
provided  in  well- annotated  source  code. 

Just  as  all  plans ,  etc . ,  for  the  electronic  elements  of  the 
computer  nust  be  available  for  all  defense  related  conputing 
equipment  to  avoid  dependence  upon  one  individual  or  ccopany  in 
a  national  emergency  (Reconmendation  9) ,  likewise  a  well-annotated 
source  code  for  all  computer  programs  must  be  available  to  permit 
any  well  qualified  individual  or  cctnpany  to  perform  any  necessary 
maintenance  or  modification  on  the  software  of  the  system.  The 
Army  cannot  afford  to  be  completely  dependent  on  a  single  vendor 
for  this  capability. 

12.  Vendor  generated  support  programs ,  parameters ,  constants,  etc. , 
must  be  able  to  be  dunroed  at  layaway  time  and  reloaded  at  reactiva- 
tion time  to  achieve~tne  same  system  configuration. 

A  conplete  and  accurate  record  of  the  actual  program  existing 
in  the  conputer  system  at  any  moment  must  be  able  to  be  retrieved 
and  stored  separately  from  the  conputer  system  itself.  Then 
should  any  accident  change  the  stored  program  of  the  ccqputer,  the 
original  accurate  program  can  be  restored  by  reading  in  the  copy 
retained  separately.    This  is  especially  inportant  during  the 
dormant  period  vAien  changes  may  take  place  in  the  memory  of  the 
systan.    The  program  existing  at  layaway  can  then  be  restored  at 
reactivation,  so  that  the  corrputer  is  in  the  state  #iich  it  held 
at  deactivation. 

13 .  Record  copies  of  computer  programs .  such  as  discussed  in  Reconmen- 
dation 12,  should  be  preserved  on  magnetic  tape  as  disciissed  in 
the  paper  ent it led~^Layaway ,  Standby,  and  Reactivation  Procedures 
for  Ccmputer  ^lagnetic  Media '  contained  herein.    A  total  of  two 
separate  copies  in  different  locations  should  be  utilized.  The 
technique  known  as  1600  phase  encoded  recording  should  b"e  used. 
Strict  tenperature  and  humidity  control  should  be  maintained  and 
strict  enforcanent  of  proper  handling  techniques  must  be  carried 
out. 

For  the  foreseeable  future,  magnetic  tape  will  remain  the 
principal  technological  base  for  long  term  auxiliary  storage  in 
digital  conputer  systans.    Therefore  proper  procedures  must  be 
followed  in  the  preparation,  storage,  and  recall  of  magnetic  tapes. 
The  proper  tapes  must  be  procured  and  certified.    When  recording 
data  the  proper  technique  (1600  phase  encoded  recording)  should  be 
used,  and  care  must  be  exercised  in  handling.    During  storage 
strict  tapperature  and  humidity  control  must  be  maintained.  At 
reactivation  tapes  should  be  allowed  to  adjust  to  the  taiperature 
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and  humidity  of  the  corrputer  installation.    Duplicate  copies  of  all 
data  should  be  maintained.    Details  of  these  procedural  and  policy 
matters  are  contained  in  the  paper  entitled  "Layaway,  Standby,  and 
Reactivation  Procedures  for  Cctipjter  Magnetic  Media." 


OTHER  SYSmi  CONSIDERATIONS 

14.  All  sensors ,  transducers  and  non-mechanical  modulators  should  be 
calibrated  vhen  installed  and  recalibrated  on  a  periodic  basis 
including~iiimediately  before  shutdown.    THey  sEould  be  removed  and 
stored  separately  on  deactivation,  calibrated  annually  while  in 
storage~and  recalibrated  on  reinstallation . 

The  major  requirement  for  an  accurate  process  data  handling 
and  control  system  is  for  an  adequate  maintenance  and  calibration 
of  the  sensors  used  with  the  system.    In  particular,  a  calibration 
history  of  the  device  will  show  vhen  nonlinear  changes  occur  in 
the  device  due  to  stress  or  other  occurrences.    Because  these 
devices  can  be  particularly  sensitive  to  environmental  changes  it 
is  best  to  reanove  them  from  the  field  site  and  store  them  in  a 
controlled  area  during  the  dormant  period.    Detailed  background  is 
provided  in  the  paper  entitled,  "Environmental  Devices  and  Equip- 
ment Under  Long  Storage  Conditions"  contained  herein. 

15 .  Current  ARRCCM  procedures  for  the  handling  of  the  mechanical 
elements  of  the  plant  control  systems ,  such  as  valves ,  solenoids , 
etc. ,  should  be  enforced.    They  appear  adequate  to  handle  most 
foreseen  situations  vhich  may  occur  during  layaway  and  the  dormant 
period. 

In  the  views  of  the  Workshop  participants  the  requirements  of 
Army  Regulations  and  ARRCOM  Procedures  as  outlined  during  the 
Workshop  discussion  appeared  entirely  adequate  for  the  layaway 
problon  as  envisioned  by  them.    Following  are  the  regulations 
and/ or  procedures  that  were  outlined. 

ICEL  SID  107E  -  Preparation  and  Handling  of  Industrial  Plant 
Equipment  for  Shipment  and  Storage 

-  Preparation  at  Industrial  Plant  Equipment 
for  Storage  and  Shipment 

-  Methods  of  Preservations  -  Packaging 

-  Amty-  Industrial  Preparedness  Program 

-  Packaging  and  Packing  of  Electronic  and 
Electrical  Equipment 


TM  38-260 

MIL-P-116F 
AR  700-90 
MIL-E-17555 
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It  is  recornnended  that  these  Regulations  and  Procedijres  be  enforced. 


16.  Electrical  power  and  signal  cabling  appears  to  have  a  virtually 
unlimited  life  if  properly  handled.  The  major  problems  are 
corrosion  of  contacts  and  connectors  and  attack  on  insulation  by 
rodents .  The  former  can  be  prevented  b^.  encasing  all  connectors 
in  plastic  bags'  along"~wrtE~a  dessicant  T^hich  is  renewed  periodi- 
cally. Cables  should  be  protected  frcoTrodents  by  conduit  or  by 
removal  and  storage  in  a  safe  area. 

Potentially  one  of  the  more  difficult  topics  of  concern  in  the 
layaway  of  plant  control  systems  and  power  systems  is  that  of  the 
cabling  involved  because  of  its  extended  nature.    Fortunately  pro- 
cedures are  well  established  for  caring  for  the  contacts  and 
connectors  involved  as  outlined  above.    Likewise  the  cCTimon  use  of 
caiduit  for  safety  purposes  also  protects  the  cabling  from  rodents, 
the  most  ccmnon  causes  of  damage. 


PERSO>JNEL 

17 .  Because  of  the  very  short  period  planned  for  reactivation  of  dormant 
plants ,  it  is  imperative  that  training  programs  be  developed  for  all 
critical  operating  and  maintenance  skills  necessary  in  the  plant. 

Since  reducing  reactivation  time  directly  reduces  the  munitions 
inventory  cost,  and  the  skill  level  of  the  operators  is  a  signifi- 
cant determinant  of  reactivation  time,  it  is  absolutely  essential 
that  personnel  be  highly  trained.    A  cadre  of  plant  operating  and 
maintenance  personnel  must  possess  all  of  the  critical  skills  re- 
qioired  to  reactivate  and  operate  the  plant.    These  skills  must  be 
maintained  during  peacetime,  vhen  demand  for  munitions  is  relatively 
small.    They  must  be  maintained  by  existing  personnel  and  acquired 
by  new  personnel,  since  normal  attrition  will  necessitate  their 
recruitment . 

Therefore,  it  is  inperative  that  a  training  program  be 
developed  for  all  critical  operating  and  maintenance  skills. 
Responsibility  for  planning  and  inplementing  such  a  program  should 
reside  at  a  high  level  of  the  organization.    Considerations  that 
should  go  into  the  plan  include:    1)  identification  of  critical 
skills,  (it  was  assumed  that  non- critically  skilled  labor  can  be 
acquired  at  reactivation  time),  2)  required  levels  of  peacetime 
personnel,  3)  mechanisms  for  incorporating  periodic  recycling  into 
the  plan  (see  Recamiendation  7) ,  and  4)  techniques  for  making 
effective  use  of  computer-based  systems  such  as  Computer-Aided 
Instruction  and  Simulation  (see  the  paper  "Use  of  Simulation  and 
Other  Related  Techniques  for  the  Training  of  Startup  Crews  for 
Reactivation  of  Laid- Away  Plants"  contained  herein,  and  Recctimenda- 
tion  18). 
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The  technical  and  econcnnic  feasibility  of  incorpor at in^.  simulation 
capabilities  in  on-line  systems  as  an  aid  to  reactivation  operator 
training  should  be  investigated. 

Simulation  was  described  following  Recomnendation  6  as  the 
duplication  of  the  behavior  of  a  given  system  acooirplished  by 
developing  a  model  of  that  systan.    This  duplicated  behavior  can  be 
used  as  the  basis  for  operator  training.    During  periods  of  train- 
ing the  operator  interface  is  unchanged,  but  the  actual  process  is 
replaced  by  a  cortputer-inplemented  model  of  that  process.  Simula- 
tion is  particularly  well-suited  for  the  Amy's  training  program 
because  it  permits  "hands  on"  experience  without  producing  any 
products . 

The  discussion  following  Recomnendation  6  pointed  out  that 
simulation  can  be  an  expensive  technique.    Thus,  the  economic 
feasibility  of  its  use  must  be  investigated.    Nonetheless,  the 
value  of  on-line  simulation  as  a  training  technique  for  operators 
has  been  demonstrated  by  many  industrial  conpanies,  and  it  appears 
well- suited  for  the  Array's  training  requirements. 


XX  ix 


PAPER  IIUMBER  1 


PAST  EXPERIENCE  IN  THE  REACTIVATION 
OF  ARMY  AMMUNITION  PLANTS 


Mr.  Augie  J.  Zahatko 
Chief  of  Production  Base  Concepts  Office  ARRCCM 
Headquarters,  U.S.  Amy  Armament  Material 
Readiness  Comnand  DSAR-INC 
Rock  Island,  Illinois  61201 


1 


PAST  EXPERIENCE  IN  THE  REACTIVATION  OF  ARMY  AMMUNITION  PLANTS 


INIRODUCnON 

The  information  and  data  in  the  following  pages  will  provide 
insight  into  problems  experienced  by  the  Government  and  its  Govemment- 
C^^ned  Contractor-Operated  (GOCO)  plants  during  reactivation  for  the 
Southeast  Asia  conflict. 

Ihis  information  is  a  conpilation  of  inputs  from  the  Amy  aracoLini- 
tion  plants,  recalling  on  the  laicwledge  and  experience  of  those  who 
were  present  during  reactivation,  and  also  contains  extractions  from 
Profile  on  Munitions  by  R.  J.  Hamnond. 

Ihe  GOCO  plants  are  groi4)ed  into  four  categories  based  on  their 
production  capability.  The  groi:ps  are  Propellant  &  Explosive  (P6E) ; 
Metal  Parts  (MPTS)  ;  Small  Arms  (SA) ;  and  Load,  Assecdble  and  Pack  (LAP) . 

After  World  War  II,  the  GOCO  ccoplex  was  maintained  on  a  reduced 
scale  so  that  at  the  time  of  the  Southeast  Asia  (SEA)  buildup  in  1965, 
eleven  plants  were  still  active.    Full  reactivation  of  26  plants  re- 
quired an  average  of  seven  months  and  a  cost  less  than  $300  million. 
Chart  1  reflects  the  biaildup  leadtimes  by  various  categories  of  opera- 
tion for  Southeast  Asia. 

Prior  to  1965,  the  level  of  production  was  mainly  peacetime  re- 
quirements for  training,  and  manufacture  of  a  few  of  the  sophisticated 
items  coming  out  of  development  for  phasing  into  the  inventory. 

It  was  not  until  the  introduction  of  American  combat  troops  into 
Vietnam  that  a  more  intensive  effort  was  given  to  reactivation  and/or 
modernization  of  the  anmunition  complex. 

Chart  2  reflects  the  history  of  reactivation  of  Anr^^  anmunition 
plants  following  our  entry  into  SEA.    As  the  war  began  to  build  in 
intensity,  six  additional  plants  ware  brou^t  to  active  status  in 
FY  66.    Eight  more  plants  were  phased  in  during  FY  68.    By  this  time, 
the  capability  to  produce  the  "bread  &  butter"  type  items  that  were 
necessary  and  critical  to  troop  sij^jport  in  SEA  was  all  being  utilized. 
There  was  only  one  plant,  Alabama  Arny  Anmunition  Plant,  that  remained 
in  an  inactive  status.    It  has  since  been  declared  excess. 

The  GOCO  ccoplex,  although  smaller  in  terms  of  plants  and  per- 
sonnel than  its  predecessors  of  World  War  II  and  Korea,  was,  on  31 
Dec  68,  still  a  vast  enterprise  vMch  spread  from  New  Jersey  to 
California  and  from  Texas  to  Nebraska  with  a  peak  contractor  employ- 
ment level  in  excess  of  120,000.    The  capacity  utilization  of  our 
GOCO  plants  during  SEA  is  reflected  in  Chart  3.    It  should  be  noted 
that  100%  of  the  LAP  capability  had  been  reactivated  and  96%  was  being 
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utilized,  1007o  of  the  metal  parts  capability  had  been  reactivated  and 
94%  was  being  utilized,  857o  of  the  e2q)losive  capacity  had  been  reacti- 
vated and  897o  was  being  utilized  (the  157o  that  remained  inactive  was 
the  Alabama  Amy  Aumunition  Plant,  vhich  had  partial  TNT  lines  avail- 
able).    In  propellants,  927o  was  active  with  70%  utilization.    The  8% 
remaining  inactive  was  also  located  at  the  Alabama  Array  Aumunition 
Plant  vMch  had  single  base  lines  partially  constructed  after  Korea. 

By  1969  it  became  readily  apparent  that  it  was  not  only  essential 
to  maintain  current  capability  but  also  to  modernize  and  expand  the 
production  base.    It  was  realized  that  the  production  of  qualitatively 
si:5)erior  amnunition  on  a  timely  basis  was  significantly  affected  by 
the  lack  of  availability  of  advanced  manufacturing  processes,  techni- 
ques, and  new  equipment.    The  continued  advancement  of  the  state-of- 
the-art  had  to  becon^  an  integral  part  of  our  efforts  to  retain  and/or 
build  an  effective  mobilization  base  that  wDuld  be  responsive  to  any 
future  military  contingencies. 

It  is  noted  that  at  the  time  of  reactivation  of  Southeast  Asia 
(1965-67)  ,  most  of  the  Amy  ammunition  plants  did  not  have  "automated 
process  control  systems"  in  layaway  status.    Therefore,  the  reactiva- 
tion experiences  are  not  totally  cooparable  to  that  of  the  present  day 
situations . 


mSOR  AREAS  OF  EXPERIENCE  AND  PROBLEMS 

The  problems  experienced  during  the  reactivation  of  the  produc- 
tion base  in  support  of  the  Vietnam  operation  were  many  and  varied; 
njnning  the  full  range  of  labor,  social  economic,  political,  geographic, 
and  technical.    Selected  problems  and  selected  plants  will  be  high- 
lighted in  this  paper.    The  problems  and  plants  to  be  discussed  are 
believed  to  be  typical,  but  may  not  necessarily  be  the  most  critical  or 
most  inportant  vhen  viewed  from  individual  points  of  interest. 


PLANNING  FOR  REACTIVATION 

One  of  the  major  problems  in  planning  for  reactivation  was  the 
uncertainty  of  the  changing  position  the  United  States  was  to  take  in 
the  overall  operation.    This  made  it  nearly  iirpossible  to  get  clear 
and  concise  guidance  as  to  production  requirements  and  the  anticipated 
duration  of  the  operation.    It  also  made  orderly  planning  extremely 
difficult.    This  resulted  in  piecemeal  reactivation,  delayed  full 
production,  and  increased  the  overall  cost  of  reactivation  and  initia- 
tion of  production. 

Another  major  problem  in  planning  was  the  mobilization  planning 
did  not  match  the  type  of  war  the  United  States  was  called  upon  to 
si5)port.    Plans  were  often  developed  on  short  notice  and  on  a  crash 
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basis  to  meet  diaiiging  delivery  requirements  and  asset  posture.  The 
resultant  reactivation  was  often  for  production  of  items  not  previously 
included  in  the  plant's  mobilization  plan.  Experience  tells  us  that 
the  mDSt  meticulous  planning  most  likely  will  not  meet  the  actual 
situation  ^toi  we  are  called  upon  to  reactivate/ expand  the  production 
base. 


ORGANIZING  AND  STAFFING  FOR  REACTIVATION 

Considerable  management  effort  was  required  to  develop  and  justify 
organizational  structures  since  there  were  no  pre-determined  plans 
to  expand  the  headquarters  (HQ)  or  to  recruit  personnel  to  si;5)port  the 
reactivation  process  and  manage  those  problems  arising  from  increased 
production.    This  effort  detracted  from  the  primary  jobs  of  reactivat- 
ing plants  and  delivering  amnunition  to  the  troops. 

Recruitment  problems  were  caipounded  by  the  lack  of  suitable 
housing  at  reasonable  prices,  unwillingness  to  accept  positions  at 
vhat  appeared  to  be  a  tenporary  installation,  and  the  grade  structure 
of  jobs  ^AMch  were  not  in  line  with  standards  required  to  qualify  for 
jobs. 


FUNDING 

Funding  was  limited  and  often  provided  on  an  incremental  basis  due 
to  the  nature  of  the  Ibited  States  conmitment .    There  were  a  few  plants 
that  did  not  experience  any  funding  problems  vMle  others  fitted  into 
one  or  more  of  the  following: 

1.  Adequate  funds  to  cover  the  general  scope  of  vrork  for  reactiva- 
tion were  furnished  on  a  timely  basis  but  subsequent  funds  for 
production  were  slow. 

2.  Funding  required  for  reactivation  was  insufficient  because 
programned  buildup  was  cccpressed  into  a  shorter  timeframe 
than  estimated.    Production  si:5)port  funds  were  inadequate  for 
the  same  reason. 

3.  Funds  required  for  training  and  debugging  were  short  and  over- 
age expense  was  later  charged  to  end  items  and  resulted  in 
higher  production  costs. 

4.  Confusion  existed  in  the  type  of  funds  to  utilize.    It  became 
necessary  to  reactivate,  convert,  and  produce  the  end  item 
with  the  available  funds. 
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GUIDANCE 


Guidance  v/as  a  problem  for  HQ,  the  Government  staff,  and  the 
operating  contractor  because  of  uncertainties,  constantly  changing 
requirements ,  and  inexperienced  people  at  all  levels . 

The  most  serious  problem  in  guidance  vTas  that  seme  of  the  plants 
were  activated  to  produce  items  which  were  not  on  their  mobilization 
schedule  and  some  items  had  not  been  previously  produced  at  the  plants . 

In  many  cases,  initial  guidance  was  lacking  except  for  direction 
to  get  into  production  as  quickly  as  possible.    Needs  were  made  known 
but  the  "how"  to  fulfill  the  needs,  and  "how"  to  reactivate  were  lack- 
ing.   The  prime  objective  was  production  without  regard  to  cost. 

Indecision  regarding  the  exact  configuration  of  itans  to  be  pro- 
duced delayed  production  as  did  late,  inadequate,  and  scmetimes  unavail- 
able technical  data  packages. 


STATE  OF  MAINTENANCE/DETERIORATION  OF  FACILITIES 

The  condition  of  the  facilities,  upon  reactivation,  was  dependent 
upon  the  layaway  status,  the  length  of  layaway,  and  the  amount  and 
type  of  maintenance.    Standby  maintenance  funding  constraints  during 
layaway  were  a  major  contributing  factor  in  the  deterioration  of 
facilities  and  the  high  cost  to  reactivate.    Some  facilities  were  in 
fair  condition  but  required  modernization,  others  were  in  very  poor 
condition  and  extensive  repairs  were  needed  to  restore  to  operative 
standards,  ^ile  others  were  damaged  beyond  repair.    Equipment  condi- 
tion codes  on  record  could  not  be  relied  upon.    Equipment  in  many 
instances  was  of  World  War  II  or  Korean  War  vintage,  was  not  easily 
adaptable  to  automation,  and  many  revisions  were  required  to  meet 
safety  standards.    Equipment,  motors,  electrical  systems,  building 
interiors  and  exteriors,  storage  tanks,  conveyor  rollers,  air  receivers, 
control  systems,  railroads,  roads,  and  distribution  systans  and  other 
items  were  damaged  and  deteriorated  and  required  repair  or  replacement. 


ACQUISITION  OF  MATERIALS,  SUPPLIES,  &  TOOLING 

TO  SUPPORT  REACTIVATION  &  PRODUCTION 

The  acquisition  of  materials,  equipment  and  supplies  to  support 
reactivation  was  hard  to  manage  because  many  people  were  looking  for 
tiie  sane  items  at  the  same  time.    Lead  times  were  constantly  extended  and 
promised  delivery  dates  were  constantly  missed. 

Much  of  the  equipment  needed  to  support  reactivation  had  to  be 
obtained  from  other  plants,  purchased,  and/or  built.  Government 
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furnished  material  (GEM)  was  in  poor  shape  since  most  of  it  was  excess 
or  salvaged  from  other  plants.    RebuLldirig  and  maintenance  of  GFM 
increased  reactivation  costs.    Several  producticn  lines  were  canni- 
balized to  support  requirements  elsevAiere. 

Some  problems  in  the  acquisition  of  materials  were  attributed  to 
an  extremely  limited  staff  of  personnel  qualified  to  develop  the 
general  and  specific  material  requirements,  including  descriptions 
and  qualities.    For  example:    At  Kansas  AAP  most  of  the  tooling  re- 
quired to  produce  the  mobilization  planned  items  was  on  hand  and  ready 
for  use.    However,  Kansas  did  esqjerience  problems  in  acquiring  tooling 
for  the  new  items  relative  to  inconplete  technical  data  packages, 
shortage  of  qualified  personnel  to  properly  interpret  requirements, 
and  using  tool  shops  of  unproven  ability.    Inconplete/outdated  techni- 
cal data  caused  delays  and  shifts  in  schedules.    Kansas  had  retained 
some  long  leadtime  items  during  standby  that  were  not  required  but 
proved  necessary  at  reactivation;  these  were  parts  for  air  carpressors, 
deluge  systene,  explosion  proof  disconnects,  switches,  tooling,  presses 
and  drills. 

The  acquisition  of  materials  and  tooling  at  Joliet  AAP  became  very 
critical  in  meeting  startup  and  production  schedules  due  to  unavailable 
current  Bills  of  Material,  outdated  and  unavailable  technical  data  pack- 
ages, inability  of  material  vendors  to  meet  requirements  and  unavaila- 
bility of  metal  parts  and  ciiponents;  i.e.,  cartridge  cases,  projectiles, 
fiber  containers,  primers,  and  propellants. 

At  Comhusker  AAP  the  supply  of  botnb  casings  controlled  production. 
MDst  stocks  of  casings  had  been  disposed  of  and  bonb  manufacturers  no 
longer  had  tooling  or  the  capability  to  make  casings. 

TMn  Cities  experienced  considerable  difficulty  in  the  acquisition 
of  plant  procured  raw  materials,  such  as  case  ci4)s,  bullet  jacket  cups, 
etc.     Blanking  and  cupping  facilities  were  limited  so  it  became 
necessary  for  the  plant  to  procure  brass  stripping  and  acccoplish 
blank  and  cupiping  operations.    GBK  shortages  of  clips,  links,  bandoliers, 
amnunition  boxes ,  etc . ,  inpacted  packing  schedules .    The  most  serious 
problem  at  "Rdn  Cities  was  procurement  of  perishable  tools  ^^»Mch  re- 
sulted from  a  conbination  of  no  technical  data  package  or  material 
specification,  inaccurate  tool  design,  limited  (often  incapable) 
procurement  sources ,  limited  sources  of  raw  material  (Tungsten 
carbide,  high  carbon  alloys),  tool  revisions,  lack  of  control  of 
engineering  changes,  and  inexperienced  personnel. 

Generally,  the  acquisition  of  tooling  and  materials  at  all  plants 
was  extremely  difficult.  Leadtimes  were  abnormally  long,  major  sup- 
pliers were  unable  to  fill  orders,  quality  and  workmanship  were  fre- 
quently below  standard,  and  late  deliveries  encroached  on  time  required 
for  debugging  and  starti:^).  Acquisition  was  further  conoplicated  by  a 
limited  list  of  vendors  capable  of  sij^plying  items  of  material  asso- 
ciated with  the  production  of  amnunition. 
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RECRUITMEM'  AND  TRAINING 


Generally,  recruitment  of  unskilled  personnel  presented  no  parti- 
cular difficulty.    A  high  percentage  of  female  help  was  utilized  and 
most  of  the  local  areas  produced  more  than  adequate  nurrbers  of  appli- 
cations for  production  vjorkers ,  laborers ,  and  clerical  positions .  How- 
ever, recruitment  problems  were  experienced  in  hiring  skilled  workers, 
graduate  engineers,  professionals,  supervisory,  and  administrative 
personnel  .    Since  most  of  the  new  hires  were  unfamiliar  with  anmuni- 
tioi.  pro  action,  extensive  training  was  conducted  both  by  classrocm 
instructi:)n  and  on-the-job  training.    Training  was  hanpered  by  a  lack 
of  updateu  methods  and  procedures,  experienced  instructors,  training 
equipment  and  materials,  funds,  space,  and  time. 


PRIVATE  SECTOR 

The  private  sector  was  not  reactivated  in  the  classical  sense  but 
experienced  many  of  the  same  problems  that  have  been  discussed;  i.e., 
deteriorated  equipment,  lack  of  technical  data,  missing  parts,  unavail- 
able material  and  inexperienced  personnel. 


SUM^Y 

As  can  be  seen  from  the  previous  discussion,  the  problems  of 
reactivation  are  varied  and  cctqplex  and  in  most  cases  directly  relate 
to  the  previous  preparation  that  has  been  acconplished.    Major  problems 
are  funding,  availability  of  trained  personnel,  ip-to-date  mobilization 
plans,  item  and  process  documentation,  availability  of  spare  parts  and 
raw  materials ,  and  physical  plant  and  equipment  deterioration  due  to 
inadequate  layaway  and  maintenance.    The  introduction  of  new,  modem, 
sophisticated,  single  purpose,  automated  production  and  process  control 
equipment  cctipounds  the  effect  of  each  of  these  factors.    The  new  sys- 
tems and  the  effect  of  each  factor  must  be  given  proper  consideration 
in  planning  for  mobilization. 
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CUEEEOT:  PR0CE3DURES  At©  PROBLEMS  IN  THE 
lAYM^JAY  AND  REACTIVATION  OF  PROCESS  CONTROL  SYSTEMS 


INTRODUCTION 

In  t±ie  early  1970' s,  an  extensive  effort  ms  initiated  to  modernize 
(lOD)  the  anmunition  production  base.    One  of  the  main  objectives  ms  to 
red-Qce  production  costs  thru  automating  production  processes.    In  the  mid 
1970 's  many  ^DD  projects  reached  cccpletion  and  it  vas  found  that  the 
teclinique  of  laying  away  (LWY)  control  systems  in  a  way  that  M)uld  permit 
rapid  reactivation  needed  further  inprovenent . 

Resolution  of  this  problem  area  is  icnportant  since  we  want  to  pro- 
tect our  past  investments;  we  warit  to  assure  mobilization  readiness;  and 
we  want  to  modify  as  necessary,  those  modernization  and  e5q)ansion  projects 
currently  in  the  planning  stage. 

ARMY  GUIDAInICE 

Existing  guidance  on  the  LWY  and  reactivation  of  control  systems 
dates  frcm  the  years         control  systems  were  sinpler.    It  does,  however, 
recognize  many  inportant  problems:* 

MDisture  -  "...  variable  and  adjustable  resistors  may  be  effected 
because  of  ejcposed  metal  surfaces;  various  kinds  of  capacitors  may  be 
damaged  by  moisture,  duist,  leaking  electrolyte,  etc.;  inductors  may 
absorb  moisture  in  their  windings;  transformers  have  a  tendency  to  rust; 
alternate  heat  and  cold  will  cause  insulation  deterioration;  connectors 
and  sockets  will  rust  and  corrode  from  moisture;  relays,  timers,  sole- 
noids, and  switches  suffer  from  corrosion  on  contact  points  and  fungus 
growth  on  coils;  and  wires  and  cables  are  damaged  due  to  chemical  attack 
on  the  insulation  frcm  contact  with  organic  solvents." 

Solvents  -  ". .  .Extreme  care  must  be  exercised  in  the  selection  of 
a  cleaning  material  to  prevent  damage  to  parts.    Solvents  (and  their: 
vapors)  may  cause  dimensional  or  chemical  changes  to  insulations,  wiring, 
or  other  susceptible  organic  parts  of  critical  electronic  equipment." 

Preservaties  -  "Petroleum  (P-type)  preservatives ..  .may  destroy  the 
usefulness  of  an  item  due  to  the  difficiAty  of  their  removal.    A  pre- 
ser\7ative  may  penetrate  into  unwanted  areas  and  cause  swelling  or  decon- 
position  of  the  material,  or  reduce  its  electrical  conductivity..." 


^Quotations  from  T14  38-260,  Preparation  of  Industrial  Plant  Equipment  for 
Storage  or  Shipment. 
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The  reccomended  inetiiDds  of  cleaning,  preservation  and  storage  are: 

Cleaning  -  "...Low  pressure,  dry,  prepared  compressed  air,  vacuurn 
cleaning,  or  wiping  with  a  clean  lint  free  cloth  may  be  used  for  cleaning 
. .  .Never,  under  any  condition,  use  abrasive  materials  for  cleaning 
electronic  cctiponents. . ." 

Preservation  and  storage  -  Since  at  least  March  1965,  it  has  been 
specified  that  contact  preservatives  which  require  reinoval  before  re- 
activation shall  not  be  used  on  coiputers,  controllers,  and  printed  cir- 
cuit boards.    Specifications,  however,  prescribe  reinoval  of  water  vapor 
from  the  storage  environment.    In  environmentally  controlled  storage,  the 
relative  humidity  is  maintained  at  50%  or  below. 


CYCLING 

Cycling,  or  exercising  used  to  be  a  popular  method  of  LVJY.  Hydrau- 
lic presses  were  nearly  always  cycled.    It  was  done  to  prevent  peening  of 
areas  on  bearings,  sb.afts,  etc.;  to  recirculate  lubricants,  hydraulic 
fluids,  etc. ;  and  to  verify  operability. 

Sijb sequent ly,  it  w^as  realized  that  cycling  merely  "rubbed  off  cor- 
rosion" (which  then  contaminated  the  lubricating  oil)  ;  that  it  was  ex- 
pensive; and  that  it  was  no  substitute  for  the  proper  cleanip.g  and  pres- 
ervation (including  blocking  to  relieve  concentrated  loads,  using  addi- 
tives in  hydraulic  and  lubricating  fluids ,  etc . ) . 

The  issue   of  cycling  has  cctne  i:ip  again  in  connection   with  the  IX>JY 
of  process  controls.    Some  have  proposed  periodic  cycling  for  several 
days  at  a  time  to  verify  operability.    A  Computer  Sciences  Corporation 
(CSC)  study*,  for  exanple,  has  recctimended  that  the  TNI  control  systenri 
at  Volunteer  and  Joliet  Anny  Antnunition  Plants  be  cycled  as  follows : 

"Once  every  tvro  years  turn  on  electrical  power  to  the  control  sys- 
tem for  four  or  five  days,  and  run  diagnostic  checks  of  all  digital  and 
analog  equipment.    Repair  all  failures  and  turn  off  power." 

The  major  factor  that  determines  cycling  frequency  is  the  number  of 
interrelated  failures.    In  ccoputer  inaintenance,  the  corputer  is  generally 
used  as  its  own  diagnostic  tool.    During  normal  operations,  failures 
appear  one  at  a  time,  and  it  is  relatively  easy  to  find  one  fault. 
During  storage,  however,  tvro  or  more  failures  may  develop  which  will  be- 
cane  simultaneous  failures  upon  reactivation.    Simultaneous  failures  may 
interact  and  the  amount  of  time  to  find  and  correct  several  interrelated 
faults  increases  at  an  exponential  rate.    Cycling  should  be  scheduled 


^Contract  IM^er  DAAAl21-75-C-0244,  "Reccmnended  Approach  to  La3rmg  Away 
PCP-88  Control  System  Electronics  at  CIL  TNT  Plants". 
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then  before  too  many  interrelated  faults  accumulate. 

A  minor  factor  that  determines  cycling  frequency  is  the  thermal 
stress  caused  by  power  on-off  cycling.    To  determine  the  effect  of  cy- 
cling on  control  systons,  Martin  Marietta^'  has  looked  into  the  question 
of  thermal  stress.    It  concluded  that  ". .  .it  appears  that  a  single  povver 
on-off  cycle  to  thermal  equilibrium  and  back  to  room  anibient  tenpera,ture 
is  between. .  .165  to  375  (depending  on  component  mix)  . .  .times  more  stress- 
ful . . .  than  one  hour  of  dormant  time .    Another  way  of  saying  thds  is  that 
one  would  expect  33  power  on-off  cycles. .  .to  be  eq\iivalent  to  just  over 
one  year  of  dormancy  in  precipitating  failures."    In  accordance  with  this 
conclusion,  too  much  cycling  would  not  only  be  more  expensive,  but  woiold 
contribute  to  the  deterioration  of  the  control  system.     (It  should  be 
noted  here  that  power  on-off  cycling  was  found  to  be  much  more  stressful 
and  it  generates  more  failures  than  continuous  operation  in  a  fully 
energized  state) . 

The  theoretical  rate  of  dormant  failures  can  be  determined  for  each 
control  system.    In  one  method,  a  physical  count  of  each  cccponent  irust 
be  made  and  multiplied  by  the  dormant  failijre  rates  specified  in  MEL- 
HDBK-271B'^.    Since  this  data  is  only  for  controlled  environment,  it  has 
to  be  modified  Vi^ien  the  humidity  and  tenperature  are  uncontrolled. 


OBSOLESCENCE 

Several  government  p\jblications  indicate  that  the  economic  service 
life  of  electronic  data  processing  equipment  is  shorter  than  metal  work- 
ing and  general  process  equipment.    The  PCP  88  cctrpj.ter  such  as  at 
Joliet  and  Volunteer  AAP's,  is  no  longer  manufactured.    Does  this  mean 
that  the  process  controls  we  already  have  should  be  replaced,  or  that  we 
should  postpone  the  purchase  of  new  control  systems  until  after  M-day? 
The  CSC  study  indicates  that  even  in  active  coipjters,  physical  wear  is 
not  a  factor  in  obsolescence  even  though  continuous  use  or  periodic  cy- 
cling wDuld  scmewhat  increase  failure  rates.    In  the  10  to  15  years 
since  computers  have  been  applied  to  process  control,  the  usual  cause  for 
replacing  the  cccputer  has  been  that  the  requirements  have  changed. 
Users  wanted  more  calculations,  more  inputs  and  outputs,  more  background 
capability  which  could  not  be  provided  by  the  old  machine.    In  terms  of 


^"Dormancy  and  Power  On-Off  Cycling  Effects  on  Electronic  Equipment  and 
Part  Reliahility"  prepared  by  1-Iartin  Marietta  Aerospace,  August  1973, 
AD  768-619. 

*^<MIL-HDBK-217B,  "Reliability  Prediction  of  Electronic  Equipment", 
20  Septanber  1974. 
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the  requirements  for  vhich  it  was  purchased,  the  corrputer  was  still  a 
usefLil  tcx)l. 

Obsolescence  does  icply  though  that  it  could  be  difficult  to  find 
an  alternative  user  for  the  conputer  during  periods  of  LWY.  Universities, 
for  exanple,  might  prefer  their  students  to  work  v/ith  more  modem  con- 
puters.    Other  potential  users  could  run  into  costly  conversion  problons, 
or  may  not  want  a  computer  that  won't  be  available  if  the  amnunition 
plant  is  reactivated.    It  is  possible  though  that  a  qualified,  long  term 
user  could  be  found  for  the  general  purpose  digital  computer.  Field 
equipment,  digital  and  analog  input  and  output  circuits,  line  panels  and 
operator  consoles  would  still  have  to  be  maintained  with  government  funda 

Obsolescence  also  iirplies  that  the  service  organization's  main- 
tenance man  would  have  to  spend  more  and  more  time  on  the  familiarization, 
diagnosis  and  repair  of  an  obsolete  conputer.    Keeping  a  knowledgeable 
cadre  of  engineers  and  control  house  operators  during  the  LWY  period 
would  minimize  reliance  on  the  manufacturer  or  servicing  organization, 
and  would  assure  a  higlier  state  of  readiness. 

The  availability  of  replacement  parts  for  obsolete  corputers  is  also 
a  problem.    A  proposed  solution  is  the  stockpiling  of  spare  conponents 
based  on  their  frequency  in  the  systen,  their  availability,  and  their 
failure  rates  as  observed  during  operation,  or  determined   based  on  KIL- 
t©BK-217B.    Data  from  this  HDBK  indicates,  for  exanple,  that  certain 
transistors  are  15  times  more  reliable  than  others,  that  certain  capac- 
itors are  50  times  more  reliable  than  others,  etc.    In  addition  to  hav- 
ing a  higher  stockpile  of  relatively  unreliable  components,  new  designs 
could  minimize  tlie  use  of  materials  that  are  unreliable  in  use  or  quickly 
deteriorate  in  storage.    Postponing  ccsiputer  purchase  until  M-day  could 
solve  these  probleofis,  but  only  at  the  expense  of  long  leadtimes  for 
design  update,  mantif actur ing ,  proveout  and  training. 

Some\>iiat  related  to  the  topic  of  obsolescence  is  the  problem  of 
skills  t±iat  the  LWY  of  sophisticated  control  systems  create.    Even  if 
these  systems  were  proved  out,  and  used  for  production,  there  are  a 
small  number  of  people  v^o  have  hands  on  experience.    This  distinguishes 
it  fran  the  many  other  systems  the  Army  has  in  storage  vMch  have  been 
used  by  hundreds  or  thousands  of  troops  on  active  duty.    When  a  war 
emergency  occurs,  the  Army  has  on  active  duty  at  least  seme  number  of 
officers  or  non-coms        have  used  and  have  been  using  equipment  similar 
or  identical  to  the  equipment  in  storage.    These  officers  and  non-coms 
provide  the  nucleus  of  the  reactivation  and  training  teams  for  the  re- 
activated equipment.    CSC  feels  that  there  has  to  be  a  similar  group  of 
capable  people  to  reactivate  the  control  syston  and  production  facilities 
I     During  the  LWY  period,  these  people  would  perform,  the  cycling  of  control 
j     equipment  as  well  as  assist  in  the  miaintenance  and  surveillance  of  the 
general  process  facilities.    Highly  skilled  individuals  could  be  shared 
between  facilities  with  identical  control  equipment,  as  Joliet  and  Volun- 
teer AAP's. 

I 
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STORAGE  ENVTROSIMEM: 


When  humidity  control  is  desirable,  conventional  equipment  is 
usually  consolidated  for  storage  in  humidity  controlled,  pipe  supported 
polyethylene  hutments  or  in  sealed  buildings,  vtere  moisture  is  removed 
through  dynamic  dehumidification.    This  beccmes  inpossible  vtoi  the 
equipment  is  too  large  or  when  other  structural  peculiarities  prevent 
hutment  construction  or  the  proper  sealing  of  the  building.    In  these 
situations,  localized  -  as  in  control  cabinets  -  himidity  control  must  be 
considered.    Humidity  is  controlled  throijgh  heating  by  light  btilbs/elec- 
trical  heating  strips  or  throiigh  direct  removal  of  moisture  by  desiccants. 

An  alternative  way  to  protect  exposed  electric  contacts  and  wire 
terminals  is  to  use  brushable  or  sprayable  conpounds  that  need  no  re- 
moval before  reactivation.    Since  these  products  have  not  been  properly 
tested  and  approved  for  government  use,  a  project  is  being  funded  to  do 
that. 

As  far  as  the  storage  environment  is  concerned,  the  general  concen- 
sus is  to  store  electronic  control  systems  in  a  controlled  environment. 
IMntainability  and  reliability  people  at  CSC  and  Foxboro,  as  well  as 
documents  oa  the  shipping  and  storing  of  electronic  assemblies ,  all 
recoimend  environmental,  especially  humidity  control  for  assijring  the 
operability  of  the  equipment. 

Basic  failure  rate  analyses  for  electronic  parts,  such  as  in  MTL- 
HDBK-127B,  were  made  in  a  controlled  environment.    To  coipare  different 
environments,  Martin  Marietta  has  analyzed  failure  rates  in  satellites, 
on  the  ground,  and  in  submarines.    The  resxilts  were  presented  as  Location 
MDde  Factors  (IMF's)  to  be  applied  to  the  basic  failure  rates.  These 
factors  are  reproduced  below: 

NORMALIZED  DORMANCY  LOCATION  VDIE  FACTORS  FOR 
PASSIVE  AND  ACTIVE  HIGH  RELIABILITY  ELECTRONIC  PARTS 


UDCATION  MODE  FACTOR 


DORMANT 
ENVIRONMENT 

PASSIVE  PARTS 
(RESISTORS  AND 

cAPAcrroRS) 

ACTIVE  PARTS 
(SEMICONDUCTORS  AND 
MICROELECTRONICS) 

SATELLITE 

0.2* 

0.5 

GROUND  -  INSim 
CONTAINER  IN  CONTROLLED 
ENVIRCMCNT 

1.0 

1.0 

GROUND  -  NO  CCNTAINER 
IN  CONTROLLED 
ENVIRDNMENI 

1.9 

5.0 

SUBMARINE 

7.4 

13.1 

*One  failure  was  assumed  to  obtain  this  factor. 
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In  calcialating  the  failure  rates  of  control  systons  in  uncontrolled 
storage  CSC  assumed  that  the  IMF's  for  submarines  apply.    It  ms  conclud- 
ed that  digital  circuit  failures  vould  greatly  increase.  Consequently, 
cycling  frequency  wuld  have  to  be  increased,  so  that  failures  could  be 
diagnosed  a  few  at  a  time,  using  the  oorrputer  as  its  own  diagnostic  tool. 
From  CSC  calculations,  it  vias  evident  that  heating  and  air  conditioning 
the  control  house  would  cost  a  fraction  of  the  cycling/ repair  costs 
necessary  without  environmental  control. 


CONCLUSiaT 

Based  on  the  prior  discussion,  the  following  conclusions  are 
suggested: 

1.  If  existing  control  systems  have  to  be  deactivated,  they  should 
be  stored  in  an  environmentally  controlled  area. 

2.  Ccxrputers  will  continue  to  perform  the  job  for  which  they  were 
purchased,  but  as  they  age,  they  will  be  more  difficult  to 
maintain  in  terms  of  skills  and  replacement  parts. 

In  view  of  the  many  unsolved  issues  remaining,  it  is  hoped  that 
this  seminar  will  throw  scrae  light  on  maintaining  our  existing  control 
systems,  and  designing  new  control  systons  for  our  processes  currently 
under  expansion,  conversion,  or  modernization.    Some  of  the  questions 
that  need  to  be  discussed  are: 

1.  \ha.t  kind  of  failures  develop  in  inactive  ccoputers,  and  could 
they  be  repaired  to  allow  production  within  90  days? 

2.  How  realistic  are  dormant  system  failure  calculations  based  on 
individual  component  failures? 

3.  In  vihat  circunnstances  is  trickle  current,  or  "leaving  power  on" 
sufficient? 

4.  In  designing  for  LWY,  vdiich  failure  prone  materials/ conponents/ 
subassemblies  should  be  avoided?    How  could  the  diagnosis  of 
interrelated  failures  be  simplified? 
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AN  OVERVIEW  OF  PERFOFMANCE 
CDNSIIM^ATIONS  OF  SENSORY  TRANSDUCERS 


INTRODUCnON 

Hiis  paper  provides  an  introduction  into  some  of  the  considerations 
relating  to  the  performance  of  sensory  transducers.    Specific  issues  of 
interest  to  the  Vforkshop  on  the  Layaway  Problem  are  among  those  that 
have  been  little  investigated  in  the  transducer  field.    The  discussion 
i^ch  follows  draws  necessarily  upon  expertise  gained  over  the  long-term 
on  transducer  application  and  calibration  questions  as  well  as  upon  ex- 
perimental work. 

"The  application  of  modem  processes  and  manufacturing  technology 
to  the  munitions  production  task  requires . . .  the  use  of  modem  process 
control  systems.    Pote^  ^ial  increased  benef its . . . are  accoipanied  by  some 
ne!w  problems.    Little,  if  any  experience  base  exists  concerning  the 
I  lon^-term  (5,  10  years)  dormant  storage  of  classical  and  modem  (analog/ 
'  digital)  industrial  electronic,  optic,  fluidic,  and  pneumatic  conponents" 
(1).    These  excerpts  frcm  D.  A.  >for locks'  paper  "Munitions  Production 
Base  Readiness"  put  this  problem  in  a  nutshell. 

The  sensory  transducer  is  a  vital  link  in  the  process  control  sys- 
tem because  it  measures  those  parameters  of  a  process  vMch  must  be 
accurately  controlled  in  order  for  the  product  to  meet  the  stipulated 
requirements . 

The  transducer  is  often  the  weakest  link  in  a  measuring  system, 
either  because  of  its  inherently  limited  performance,  particularly  vhen 
exposed  to  severe  environments,  or  because  of  limited  knowledge  concem- 
ing  its  performance.    This  situation  results  from  tvro  major  factors.  In 
the  design  of  transducers,  adequate  information  has  not  always  been 
available  to  transducer  manufacturers  concerning  special  performance 
needs  of  users.    At  the  same  time,  adequate  information  of  the  perfor- 
mance of  transducers  has  not  been  available  to  guide  users  in  the  selec- 
tion and  application  of  transducers  (2) . 

The  measuring  transducers  (transmitters)  used  in  industrial  systems 
may  be  based  on  electronic,  optic,  fluidic,  or  pneumatic  principles. 
The  increasing  use  of  conputer  control  places  correspondiiigly  increasing 
emphasis  on  electronic  measuring  devices  or  at  least  on  those  with  an 
electrical  output.    Such  electro-mechanical  transducers  have  been  used 
extensively  in  aerospace  and  defense  research,  and  are  being  used  in 
industrial  applications  in  ever- increasing  numbers.    Although  the  fol- 
lowing discussion  deals  with  specific  devices,  the  problems  encountered 
and  needs  for  solutions  should  be  viewed  as  being  cotimon  to  the 
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application  of  other  types  of  process  control  transducers. 


OPERATING  REGIMES  j 

The  primary  requirei]3ent  inposed  on  any  sensory  transducer  is  that  c 

it  measure  the  parameter  it  was  designed  to  measure  wLtb.  the  desired  li 
accuracy. 

The  operational  capabilities  of  a  transducer  are  described  by  its  [ 

performance  characteristics,  vMch  are  in  general  obtained  from  a  series  j 

of  calibrations  and  evaluation  tests.    The  degree  to  vMch  the  opera-  I 

tional  capability  can  be  predicted  or  verified  depends  on  the  conplete-  j 

ness  of  the  knowledge  of  the  performance  characteristics,  \^4lich,  in  turn,  i 

depends  on  the  ability  of  the  calibrations  and  evaluation  tests  to  es-  > 
tablish  all  the  needed  performance  parameters. 

For  the  piirposes  of  this  paper,  the  operating  environments  in  vMch  3 
sensory  transducers  are  used  may  be  categorized  into  regimes  as  follows: 
operation  at  laboratory  conditions,  operation  in  severe  enviroiments , 
operation  over  extended  periods  of  time,  non-operating  storage,  and  I! 
operation  in  any  of  the  first  three  regimes  following  storage.    Practi-  » 
cal  experience  with  transducers  suggests  that  the  order  in  \<Mch  these  | 
regimes  are  given  represents  the  order  of  increasing  possible  degrada- 
tion of  performance  and,  most  significantly,  the  order  of  decreasing  i 
loxfwledge  of  the  actual  type  and  magnitude  of  performance  degradation.  • 


OPERATION  ^  LABORATORY  CONDITIONS 

Operation  at  laboratory  anibient  conditions,  for  short  periods, 
essentially  in  a  well-controlled  environment,  eliminates  the  need  for 
concern  over  the  effects  of  severe  environments  or  the  effects  of  long 
periods  of  operation.    The  next  concerns  are  then  the  two  major  modes 
of  measurement  \h±ch  the  transducer  is  called  upon  to  perform:  static 
(or  quasi- static)  measurements  and  dynamic  measurements. 


STATIC  CHARACTERISTICS 

The  performance  characteristics  \<^ich  determine  the  transducer's 
ability  to  measure  static  (or  quasi-static:    only  very  slowly  changing) 
quantities  are  established  by  means  of  static  calibrations.  These 
characteristics  include  such  parameters  as  sensitivity,  zero  output, 
linearity,  hysteresis,  repeatability,  resolution,  creep,  dead  band,  etc., 
(3).    Static  calibration  procedures  for  many  transducers  have  been  pub- 
lished and  incorporated  in  standards  documents  (4) ,  and  although  gaps 
exist  in  scxvb  areas,  tiie  general  field  of  static  calibration  for  trans- 
ducers is  covered  reasonably  well. 
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DYNAt^C  aiARACEERISTICS 


With  rare  exceptions,  sensors  are  called  upon  to  measure  changing 
quantities.    It  is  necessary  not  only  to  know  the  amplitude  of  the 
quantity  at  a  given  time  but  how  it  is  changing  at  that  time,  since  the 
latter  may  require  rapid  corrective  action. 

If  the  sensor  is  to  be  used  to  measure  (or  follow)  such  changes, 
tlie  dynamic  characteristics  of  the  device  correspondingly  must  be  known. 

j  Calibrations  to  establish  the  dynamic  characteristics  (such  as  anplitude 
response,  phase  response,  resonant  frequency,  etc.,  (5),  are  less  well 

\  developed,  in  general,  than  static  calibrations.    Those  dyamic  cali- 
brations that  are  available  tend  to  cover  relatively  narrow  amplitude- 
and  frequency-ranges  (6) . 


OTHER  FACTORS 

Even  at  laboratory  ambient  conditions,  there  are  factors  external 
to  the  sensor  ^^Mch  can  affect  its  performance.    Pov^er  supply  variations, 
electrical  noise,  mechanical  vibration,  lead  resistance  variations,  etc., 
will  tend  to  degrade  the  measuring  ability  of  tlie  sensor. 

Thus,  information  is  needed  on  a  large  number  of  performance  char- 
acteristics before  the  quality  of  measuronents  performed  by  the  sensor 
at  laboratory  anfcient  copjditions  (the  most  benign  operating  environment) 
can  be  estimated. 


OPERATION       SEVERE  ENVIRONMENTS 

The  majority  of  sensory  transducers  do  not  operate  in  a  benign  en- 
vironment. They  are  subjected  to  a  variety  of  environmental  conditions 
vAiich  can  severely  influence  their  performance. 

Such  severe  environments  can  affect  sensors  in  two  major  ways. 
Measurement  performance  can  be  adversely  affected  \\Mle  the  sensor  is 
subjected  to  the  specific  environment.    Such  changes  in  performance  are 
generally  reversible,  and  disappear  viien  the  severe  environment  no 
longer  acts  on  the  sensor.    In  addition,  the  action  of  the  environment 
on  the  sensor  may  result  in  sane  irreversible,  and  thus  permanent, 
change  in  its  performance. 

To  adequately  assess  sensor  performance,  it  is  essential  to  be 
aware  of  both  types  of  effects.    This  subject  will  be  considered  in 
the  next  two  major  headings  of  this  paper.    Environmental  parameters  that 
have  proven  to  be  of  practical  interest  are  briefly  discussed  in  the 
following  paragraphs. 
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TEMPERATURE 


Tenperature  above  and  below  laboratory  ambient  is  the  most  fre- 
quently encountered  environmental  condition  influencing  sensor  perfor- 
mance.   \v1iile  sensor  performance  normally  is  affected  by  tenperature ,  it 
is  frequently  possible  to  compensate  for  temperature  effects,  provided 
the  operating  tenperature  is  tTiown  and  constant.    This  is  true  \\^ether 
the  entire  sensor  is  at  the  operating  tenperature  or  only  a  part  of  it, 
siich  as  tlie  diaphragm,  of  a  pressure  transducer.    In  tlie  latter  case,  a 
tenperature  gradient  may  exist  between  the  sensor  and  the  surrounding 
environment,  but  vfcen  the  gradient  is  stable,  compensation  can  be  effec- 
tive. 

If  the  process  or  environmental  tenperature  varies,  tenperature 
ccopensation  cannot  be  effective.    In  fact,  conpensating  networks  within 
the  sensor  may  actually  degrade  the  output.    Tliermal  transients  can 
cause  unexpectedly  large  measurement  errors  unless  transducers  are 
selected  for  their  inpervLousness  to  them  or  are  protected  against  them 
(7). 


SHOCK  AND  VIBRATION 

As  with  temperature,  measurements  performed  by  a  sensor  can  be 
severely  affected  by  tlie  presence  of  shoclc  and  vibration  environments. 
vfeLch  may  generate  spurious  signals  and  otherwise  interfere  with  trans- 
ducer performance. 

HUMIDITY 

The  presence  of  moisture  may  cause  undesirable  shunting  of  resis- 
tive and  capacitive  elements,  as  well  as  current  leaks  to  ground.  Viore 
inportantly,  however,  exposure  to  humidity  can  result  in  corrosion  of 
maetal  parts  and  conponents  and  can  lead  to  component  failures. 

OTHER  ENVIRONMEI^nAL  PARAMETERS 

There  are  a  number  of  other  environmental  parameters  T^M-ch  may 
affect  the  measurement  performance  of  sensors,  such  as  dust  arid  sand, 
salt  spray,  high-level  acoustic  excitation,  electromagnetic  radiation, 
magnetic  fields,  and  ionizing  radiation. 


OPERATION  OVER  EXTENDED  PERIODS  OF  TIME 


In  aerospace  arid  defense  applications,  transducers  are  generally 
expected  to  perform  properly  and  reliably  over  short  periods  of  time, 
fran  minutes  to  hours .    In  industrial  applications ,  on  tlie  other  hand , 
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the  periods  of  time  over  ■vMch  these  devices  are  expected  to  operate  may 
range  fron  days  to  several  years. 

When  extended- time  operation  is  considered,  reliability  is  a 
particularly  inportant  factor.    The  reliability  of  devices  has  been 
conputed  from  knowledge  of  coiponent  failure  rates  and  has  been  applied 
to  critical  aerospace  applications  and  expressed  as  mean  time  between 
failures.    This  concept  of  reliability,  however,  is  usually  concerned 
with  the  total  failure  of  the  sensor,  not  witli  the  deterioration  of 
sensor  performance.    Hie  latter  has  received  relatively  little  atten- 
tion. 


LIFE  CYCLING  MP  DURABILITY 

A  limited  investigation  was  conducted  on  the  effects  of  a  million 
[pressure  cycles  on  the  performance  characteristics  of  several  t37pes  of 
S strain-gage  pressure  transducers.    In  these  tests,  the  transducers  were 
subj  ected  under  laboratory  ambient  conditions  at  a  rate  of  about  once 
jper  second  to  pressure  pulses  with  anplitudes  of  80%  to  90%  of  the  range 
'of  the  transducer  until  a  total  of  10^  cycles  had  been  ccmpleted.  The 
jmain  conclusions  from  this  investigation  were:    (A)  both  output  at  zero 
pressure  and  sensitivity  changed  more  during  the  first  10^  cycles  than 
si]bsequently;  (B)  after  about  10^  cycles,  the  output  zero  pressure 
jhad  shifted  about  TA  of  the  full-scale  range  and  the  sensitivity  dif- 
fered by  as  much  as  0 . 57o  frcm  its  initial  value ;  and  (C)  linearity  and 
[hysteresis  changes  due  to  cycling  are  minor  ccoipared  to  the  changes  in 
iZero  output  and  sensitivity  (8). 

I        A  more  extensive  investigation  was  conducted  on  a  group  of  bonded- 
jjwire  strain  gage  pressure  transducers.    These  experienced  pressure 
! cycles  at  a  rate  of  five  times  per  second  under  laboratory  ambient 
i  conditions  until  40  x  10^  cycles  had  been  conpleted.    Some  of  the  trans- 
iducers  were  cycled  at  66 °C  (150°F) .    Among  the  conclusions  drawn  from 
tliese  tests  were:    (A)  the  sensitivities  of  the  transducers  tested  at 
ambient  conditions  decreased  with  cycling  as  well  as  time;  the  sensi- 
I  tivities  increased  at  the  elevated  tenperatures ;  at  zero  pressure  out- 
puts decreased  with  cycling  and  time  for  both  tenperature  regimes ;  (B) 
changes  in  hysteresis  and  linearity  were  very  small;  and  (C)  somevfiat 
larger  changes  in  characteristics  occurred  during  the  first  10^  cycles 
(the  first  three  or  four  days  of  testing)  than  subsequently  (9)  .   In  both 
investigations  cycling  at  pressures  liigher  than  full  scale  produced 
( total  failures  or  radical  changes  in  characteristics  within  the  first 
jllO^  cycles. 

'jl         It  should  be  noted  that  the  conclusions  obtained  from  both  investi- 
! I  gations  are  based  on  a  small  number  of  specimens  of  a  few  types  of  pres- 
sure sensors  subjected  to  a  particular  durability  testing  concept. 
Nevertheless,  the  test  results  point  to  the  type  of  behavior  vMch 
p  other  sensors  might  be  expected  to  exhibit.    It  is  clearly  desirable  to 
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investigate  such  behavior  further,  to  obtain  an  indication  of  possible 
problem  areas. 

NON-OPERATING  STORACT: 

Shelf  life  is  probably  the  area  of  greatest  concern  for  the  Work- 
shop, but  there  appears  to  exist  little  information  on  sensor  belriavior 
following  specif  iced  storage  conditions,  nor  has  any  experimental  vork 
been  done  at  NBS  to  address  this  special  area.  \ 

A  limited  investigation  was  conducted  on  the  effects  of  storage  at 
elevated  tenperatures  on  the  performance  of  strain-gage  pressure  trans- 
ducers.   Seven  transducers  were  si±>jected  to  tenperatures  of  107  "C  and 
three  others  to  91°C  for  a  period  of  five  weeks,  followed  by  a  tliree- 
week  period  at  laboratory  ambient  conditions.    Excitation  voltage  was 
applied  during  the  entire  test  period.    Test  results  indicated  permanent 
changes  at  the  end  of  the  test  program  of  about  0.5%  in  sensitivity  and 
up  to  4.57o  of  full  scale  in  output  at  zero  pressure.    It  should  be 
noted  the  storage  temperature  in  all  cases  was  below  the  maximum  speci- 
fied operating  temperature  for  the  transducers  tested.    The  largest 
portion  of  the  changes  appears  to  occur  during  the  first  five  days  of 
the  test  (10) .    These  conclusions  are  valid  only  for  the  actual  trans- 
ducers tested  and  may  not  apply  to  other  pressure  transducers  t5?pes. 

While  information  should  be  available  frcm  sensor  manufacturers  on 
the  shelf  life  of  transducers,  proprietary  considerations  make  it  diffi- 
cult to  obtain  this  information. 

Again,  as  in  the  other  areas  discussed  earlier,  considerable  work 
remains  to  be  done  to  explore  the  shelf-life  aspects  of  measurement 
transducers . 


ELCCTRONIC  SIQm  OONDITIONEPS 

In  the  vast  majority  of  applications,  the  electrical  output  from 
the  transducer  or  sensor  is  fed  to  seme  kind  of  electronic  signal  con- 
ditioner.   Such  a  device  may  be  used  (for  example)  as  an  anplifier  to 
raise  the  signal  level,  as  an  impedance  converter  to  match  otlier  cir- 
cuits, as  an  analog- to-digital  converter  to  provide  computer- conpatible 
signals,  or  as  a  conparison  anplifier  for  subsequent  control  purposes. 

The  electronic  siganl  conditioner  is  most  likely  to  be  a  solid- 
state  device,  either  hybrid  or  LSI  (large-scale  integrated  circuit). 
Furthermore,  the  signal  conditioner  may  be  an  intergral  part  of  the 
transducer,  or  it  may  be  remote  from  it. 

Electronic  signal  conditioners  may  also  be  affected  by  evniron- 
mental  and  storage  conditions,  particularly  in  the  case  of  long  term 
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storage.    The  specific  type  of  electronic  cctrponents,  and  their  partic- 
ular technology,  will  largely  determine  -what  changes  might  occur  with 
time.    Certain  general  considerations  apply,  however. 

Environmental  humidity  may  well  be  the  major  factor  of  concern. 
The  action  of  moisture  on  many  metals  can  lead  to  corrosion,  which  in 
turn  can  caiase  poor  contacts  at  junctions.    On  the  other  hand,  moisture 
can  also  set  up  additional  conductive  paths  across  insulators  to  create 
leakage  currents  and  short  circuits.    Extensive  wyrk  on  hermeticity  and 
sealing  problems  of  semiconductor  devices  has  been  carried  out  in  the 
Electronic  Technology  Division  of  the  National  Bijreau  of  Standards. 
Plastic  semiconductor  packages  may  be  more  prone  to  moisture  leakage 
tham  metal  cased  ones .    The  effects  of  the  normal  climatic  variations  in 
tecnperature  on  semiconductor  electronics  is  probably  minimal,  but  ther- 
mal expansion  and  contraction  can  cause  atmospheric  moisture  to  be  drawn 
into  inperfectly  sealed  electronic  packages.    Temperature- time  effects 
on  sane  insialating  materials  can  lead  to  their  deterioration    (11)  . 


OPERATION  FOLLOWING  STORAGE 

Operation  of  devices  following  storage  depends  on  the  history  of 
I    the  transd\acer  during  storage,  and  that  prior  to  storage.    Very  little 
;    information  appears  to  be  available  on  non-operating  storage.    It  is 
likely,  however,  that  organic  ccopounds  used  in  transducers  such  as 
rubber  o-rings  or  seals  and  insulation  will  deteriorate  with  time, 

I  particularly  in  the  presence  of  ozone  or  UV  light.    The  limited  ex- 

II  perimental  investigation  described  in  the  section  on  Non-Operating  Stor- 
j    age  was  conducted  basically  at  benign  laboratory  conditions  except  for 

tiie  temperature  variable.    That  is,  the  pressure  medium  was  breathing 
quality  air,  the  transducers  were  new  \^^en  the  test  started,  and  the 
duration  of  the  test  was  relatively  short.    Nevertheless  significant 
changes  were  observed.    Similarly,  shelf  storage  at  the  transducer  manu- 
'    facturer's  facility  would  probably  be  at  relatively  benign  environmental 
i    conditions  for  new  transducers,  possibly  in  closed  boxes.    This  would 
serve  to  reduce  the  influence  of  atmospheric  contaminants. 

li 

I  On  the  other  hand,  storage  of  transducers  in  situ  after  they  have 

I    been  used  for  process  measurements  is  likely  to  be  quite  different.  Ih- 
j    less  the  entire  measurement  syston  is  carefully  purged  of  all  traces  of 
I    the  measured  process  medium,  chemical  processes  could  occur  with  harmful 

effects  on  the  sensor.    Midly  corrosive  remnants  of  this  medium  acting 
^    over  the  postulated  storage  life  could  cause  substantial  change  in  sane 

sensor  caiponents  such  as  diaphragms  and  seals.    Otherwise  hannless 
I     traces  could  be  hygroscopic,  attracting  and  holding  enough  atmospheric 
j    moisture  to  accelerate  corrosion  greatly. 

REACTIVATION  CX)NSIDERATIONS 
'  It  may  be  desirable  that  \ihen.  an  industrial  process  is  reactivated 
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after  a  prolonged  period  of  layaway  the  following  steps  should  be  con- 
sidered:   First,  one  might  determine  if  the  measuring  system  is  clean 
and  if  not,  correct  the  condition.    Corrosion  or  debris  may  block  pass- 
ages, and  may  even  have  damaged  sensor  coiponents.    Second,  it  is  sug- 
gested to  check  the  functional  integrity  of  the  entire  measurement  sys- 
tem to  identify  faulty  ccqponents  for  replacement.    Third,  it  is  de- 
sirable to  carry  out  a  cctrplete  system  calibration  frcm  the  sensing  end 
of  the  transducer  through  the  entire  measurement  system  to  establish  the 
performance  characteristics  following  layaway  storage.    Fourth,  one 
might  follow  this  calibration  by  a  short  period  of  exercise:    by  apply- 
ing the  full-scale  value  and  zero  value  of  the  process  variable  (or 
suitable  test  input)  alternately  to  the  transducer  at  a  moderate  rate 
for  a  short  period  of  time  (of  the  order  of  minutes) .    Fifth,  one  might 
carry  out  a  second  ccraplete  system  calibration.    If  the  results  of  these 
two  system  calibrations  agree  with  each  other,  the  short-term  stability 
of  the  system  is  likely  to  be  satisfactory.    If  these  calibrations  also 
agree  witli  a  system  calibration  obtained  before  layaway,  then  it  is 
likely  that  the  long-term  stability  of  the  system  is  also  adequate  and 
there  is  assurance  that  this  measurement  system  will  perform  as  intended 
during  the  subsequent  process  reactivation  provided  the  operational  con- 
siderations below  are  incorporated  also. 


OPERATIONAL  OONSIEERATIONS 

It  is  not  sufficient,  however,  to  verify  only  once  the  performance 
of  any  measurement  system  that  is  to  be  used  for  any  extended  period  of 
time.    Continuing  operation  of  any  measurement  system  tends  to  lead  to 
degradation  of  performance.    Also,  following  reactivation  residual  ef- 
fects resulting  from  storage  may  gradually  degrade  sensor  performance. 

Consequently,  it  is  necessary  to  recheck  system  operation  and  trans- 
ducer calibration  at  selected  intervals.    At  present  there  seems  little 
agreement  as  to  the  proper  time  intervals  for  such  recalibrations . 
Economic  considerations  play  a  large  role  in  the  decision,  particularly 

calibration  requires  interruption  of  the  process  operation. 


CONCLUSIONS 

It  can  be  seen  that  there  are  a  large  nunber  of  factors  which  have 
a  bearing  on  the  performance  characteristics  of  transducers.    The  fac- 
tors can  be  grouped  into  (A)  those  inherent  to  the  transducer  principle, 
design  and  fabrication,  (B)  those  due  to  the  envLrorment  in  which  the 
transducer  operates,  (C)  those  due  to  the  continued,  long  term  operation, 
and  (D)  those  due  to  long  term,  non-operating,  storage.  Relatively 
little  has  been  done  to  investigate  these  factors  or  to  develop  test 
methods  or  theoretical  concepts  to  attenpt  to  predict  the  influence  of 
these  factors. 
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RECX3MME[©ATI0NS 


A  two-fold  approach  is  needed  to  assure  continoing  proper  measure- 
ment performance  on  the  part  of  transducers  and  measuranent  systems ,  in 
particular  following  layaway. 

1.  The  development  of  performance  concepts  and  test  methods  to 
verify  sensor  performance  under  the  variety  of  operational  and 
environmental  conditions  constitutes  the  major  need. 

2.  The  design  and  installation  of  measurement  systems  with  the 
capability  of  in-place  calibration  is  highly  desirable  for  any 
kind  of  industrial  process  application,  and  is  particularly 
critical  for  a  process  meeting  national  defense  needs. 
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PRDDUCriON  BASE  RESPOJSE  EEQUIEED  TO  MEET 
MDBILIZATION  REQUIEEMENTS 


INTRODUCTTON 

A  fundamental  policy  of  tiie  Department  of  Defense  is  to  maintain  an 
aninunition  production  base  cap^le  of  respondi-^g  to  emergency  production 
requirements  in  the  event  of  a  full  scale  or  limited  mobilization.  In 
general,  this  base  is  expected  to  produce  end  item  anmmition  at  a  rate 
equal  to  consimption  within  a  specified  time  period.    To  provide  this 
responsive  base  requires  a  caisiderable  amount  of  planning  with  industry, 
and  an  intensive  management  of  the  Government  resources  required  to  sup- 
plement private  industrial  capacity. 


M3BILIZATIQN  REQUIREMENTS 

First,  the  mobilization  requirements  themselves  must  be  developed. 
Due  to  the  tremendous  nunfcer  of  items  required  for  mobilization,  a 
selection  process  is  necessary  to  limit  planning  to  essential  items. 
Items  are  selected  for  planning  only  if  cne  or  more  of  the  following 
criteria  are  met: 

1.  Require  a  long  leadtime  (at  least  6  mcnths) 

2.  Require  development  of  Govemment-omed  facilities  or 
additional  capacity  to  meet  requirements. 

3.  Require  continuous  surveillance  to  insure  preservation 
of  an  adequate  base. 

4.  Require  critical  skills  or  specialized  production  equipment. 

For  those  items  selected,  base  retention  mobilization  rates  are 
generated  based  on  anticipated  ccoibat  consumption  rates  at  Df6  (6  months 
after  ccranencQnent  of  military  operations) .    It  is  sometimes  not  readily 
mderstood,  however,  that  there  are  different  mobilization  rates 
ejq^ressed  for  different  purposes.    For  exanple,  there  are  base  retention 
mobilization  rates  that  differ  from  modernization  or  investment  level 
rates.    Investment  level  rates  are  used  to  determine  the  investment  in 
facilities  required  to  support  peacetime  procurement  to  satisfy  inventory 
deficits.    Modernization  level  rates  are  lower  in  priority  and  are  used 
to  determine  the  extent  of  approved  modernization  projects  for  existing 
facilities.    These  rates  are  usually  Icwer  rates  that  are  constrained  by- 
budget  levels  and  available  funding.    Industrial  Preparedness  Planning  is 
accomplished  against  the  base  retention  level  rates. 
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M-EAY  ASSUMPTIONS 


Incliaded  witMn  the  various  plans  are  assijcnptions  that  are  made  with 
regard  to  the  anticipated  environinent  and  circumstances  that  will  exist 
on  the  day  that  mobilization  is  to  begin  (M-Day)  .    Critical  M-Day  ass\iip- 
tions  include  the  availability  of  all  required  funding  as  well  as  the 
availability  of  required  personnel.    Ihese  two  basic  assunptions  underlie 
the  plant's  estimated  mobilization  response  capability  and  are  critical 
to  the  initial  producticn  leadtirae.    Otiier  assunptions  which  might  apply 
to  a  specific  plant, e.g. ,  that  equipment    on  loan  from  standby  lines  will 
be  returned,  or  a  project  to  expand  facilities  will  be  conpleted,  may 
also  be  critical  to  the  acceleration  or  biiild-up  from  initial  production 
to  peak  production  rates  in  the  particular  situation. 


INITIAL  PRQDUCnC^  LEADTIMES 

Various  factors  must  be  given  consideration  in  determining  initial 
production  leadtime.    Generally,  conponent  production  is  scheduled  for 
MI-3  with  LAP  one  month  later.    Since  fe&7  producers  will  be  in  production 
of  the  assigned  item  on  M-Day,  the  most  cctnnDn  problem  is  to  set  up  the 
production  line.    Many  pieces  of  industrial  equipment  must  be  retrieved 
frcm  storage.    Some  will  be  located  in  Government  storage  and  must  be 
transported  to  the  production  site,  installed  and  tested  prior  to  initial 
production.    Ccmnercial  facilities  must  generally  runout  the  current 
product,  procure  materials  and  hire  and/ or  train  additicoal  personnel. 
Clearly  this  reactivation  process  must  be  well  planned  in  order  to  avoid 
problems  that  will  delay  production  bey  end  the  required  time  frame. 


SUSTAINED  PEAK  PRODUCTION 

The  build-up  to  maximum  must  be  capable  of  being  sijstained  indef- 
initely.   In  order  to  develop  a  credible  sustained  peak  schedule,  it  is 
current  practice  to  schedule  a  production  line  to  produce  at  not  more 
than  its  rated  capacity  at  500  hours  per  month  at  707o  efficiency.  For 
example,  a  line  would  not  normally  be  scheduled  to  operate  at  a  sustain- 
ed rate  on  a  3-8-6  or  3-8-7  shift  basis. 


PRIVATE  SECTOR  INVOLVEMENT 

In  accordance  with  Department  of  Defense  policy,  the  U.S.  Army 
relies  upon  private  industry,  where  feasible,  to  meet  these  wartime 
mobilization  requirements. 

In  order  to  encourage  producers  in  private  industry  to  be  prepared 
to  participate  in  the  Mobilization  Production  Program,  the  Government 
has  established  Plant  Equipment  Packages  (PEP's)  where  it  has  been 
determined  that  ownership  by  private  indistry  of  the  equipment  required 
in  the  production  of  munitiois  is  not  econctnical. 
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A  PEP  is  a  ccaiplement  of  active  and/ or  inactive  plant  equipment 
vMch  has  been  formally  approved  for  retention  as  part  of  the  najnitions 
production  base  by  the  Assistant  Secretary  of  Defense  (Installation  and 
Logistics)  and  assigned  a  number.    These  PEP  packages  conprise  a  signif- 
icant portion  of  the  industrial  base  and  cccnplement  the  Government-Owned 
Contractor-Operated  (GOCO)  and  Government-Owned  Government-Operated  (GOGO) 
Facilities . 


ORGANIZATION  AND  POLICIES/PPOCEDURES 


The  U.S.  Amy  Armament  Materiel  Readiness  Connand  (APRCCM)  has  the 
responsibility  for  integrated  ccninodity  management  of  conventional  weap- 
ons and  anmunition.    Within  ARRCQM  there  are  four  major  directorates 
concerned  with  the  management  of  the  Industrial  Base.    These  are  the 
Installations  and  Services  Directorate,  which  is  primarily  responsible 
for  facility  management  and  maintenance  other  than  Industrial  Plant 
Equipment  (IPE) ;  the  Industrial  Management  Directorate,  which  is  respon- 
sible for  the  IPE  management,  layaway  and  reactivation  of  tiie  Industrial 
Base;  the  Procurement  Directorate  vMch  is  responsible  for  contractural 
effort,  and  the  Production  Directorate  which  is  responsible  for  schedul- 
ing of  current  requirements. 

Within  the  Industrial  Managenent  Directorate,  the  Industrial 
Preparedness  Division  developes  mobilization  requirements  and  executes 
planning  agreements  with  industry.    The  Industrial  Pvesources  Branch  of 
the  Industrial  Preparedness  Division,  is  specifically  responsible  for 
equipnent  allocation  and  management,  layaway,  and  reactivation  of 
facilities . 

There  are  other  elements  in  additiai  to  ARROCM  that  interface  with 
industry.    Among  these  are: 

A.  Industrial  Base  Engineering  Activity  (IBEA)  ,  which  is  an 
agency  of  EARCCM  Headquarters  and  coordinates  responsibility 
for  the  Equipment  Management  Program. 

B.  The  Defense  Industrial  Plant  Equipment  Center  (DIPEC) 
has  the  responsibility  for  management  of  all  Defense 
Industrial  Plant  Equipment. 

C.  Defense  Contract  Administration  Service  (DCAS)  which  is 
the  onsite  liaison  organization  between  the  Procuring 
Contracting  Officer  (PCO)  and  the  private  contractor. 
DCAS  and  DIPEC  are  both  part  of  the  Defense  Supply  Agency 
(DSA). 
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D.    The  Project  Manager  of  the  Munitions  Production  Base 
MDdemization  and  Expansion  (MPBME)  located  at  Dover, 
\Aio  among  other  missions  has  the  responsibility  for  the 
PEP  Modernization  Program. 


The  ARRCm  Industrial  Base  consists  of: 


A.  Contractor-O^d  and  Contractor-Operated  (OOCD)  facilities, 

B.  Govemment-Omed  and  Government-Operated  (GOGO)  Plants, 
Production  Test  Facilities  and  Depot  Level  Maintenance 
Facilities . 

C.  Govemment-CXjned  and  Contractor-Operated  (COCO)  Plants. 

D.  Plant  Equipment  Packages  (PEP's)  capable  of  augnenting 
the  above  capabilities. 


ARRCCM  is  concerned  with  the  layaway  and  reactivation  of  GOOD, 
GOGO,  and  COCO  plants  and  with  the  management  of  equipment  located  in 
the  approximately  172  PEP's  throughout  the  nation  in  GOGO,  GOGO,  and  COCO 
plants.    IWenty-one  ARRCCM  PEP's  are  for  weapons  and  151  PEP's  are  for 
munitions  production.    Of  the  151  munitions  PEP's,  24  are  located  with 
GOGO  producers,  6  with  GOGO's,  80  located  with  private  industry  and  41 
are  designated  as  Former  Facilities  (X-Facilities) . 

The  PEP's  contain  the  Industrial  Plant  Equipnent  (IPE)  and  are 
controlled  by  procedures  established  in  AR  700-90.    Documentation  is 
prepared  by  AEROOM  and  must  be  coordinated  with  the  Assistant  Secretary 
of  Defense. 


The  PEP  includes  a  means  through  vvMch  a  contractor  can  obtain 
additional  IPE.    When  a  contractor  requires  an  item  of  Government-Owned 
Production  Equipment  to  sijpport  current  production  due  to  expansion  of 
current  production,  replacement  of  inoperable  equipment  or  to  inprove 
efficiency  of  production;  procedures  are  available  which  allow  obtaining 
replaceinent  equipnent  from  the  DIPEC  inventory.    If  no  replacement 
equipment  is  available  thru  DIPEC  new  equipment  may  be  obtained  on  loan 
or  purchase  if  jijstified. 

The  decision  to  lay  away  a  particular  line  or  PEP  is  brou^t  about 
by  a  reduction  in  requirements.    The  information  on  this  reduction 
transmitted  to  the  contractor  through  the  PCO  results  in  production 
stoppage  and  a  line  becoming  idle.    The  PCO  then  requests  the  Industrial 
Preparedness  Planning  Branch  to  provide  a  decision  as  to  whether  the  PEP 
should  be  retained  or  disbanded.    The  decision  is  based  on  the  capability 
of  the  line  and  the  current  mobilization  requirements. 

When  no  need  exists  for  further  retention  of  the  PEP,  action  to 
disband  the  PEP  is  taken.    Equipment  contained  in  the  PEP  is  allocated 
to  other  PEP's  as  required  or  released  to  DIPEC  for  redistribution  or 
disposal  as  appropriate. 
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When  t±e  decision  has  been  made  to  retain  a  PEP,  the  project 
engineer  responsible  for  the  formulation  of  the  layaway  project  determines 
costs  and  storage  location  and  processes  a  layaway  project  to  obtain  funds 
for  execution  of  the  project.    Determination  of  where  to  locate  Plant 
Equipment  Packages  is  based  on  dispersion,  the  time  required  for  reac- 
tivation, losses  v;hich  could  result  from  moving  the  equipment,  and  a 
conparison  of  costs  of  cn-site  and  other  types  of  storage.    The  final 
decision  is  consistent  with  attainment  of  maximum  econony,  essential 
state  of  mobilization  readiness,  and  the  best  interest  of  the  Government. 
Preference  is  given  in  the  following  order: 

1.  Maintain  the  equipment  in  the  facility  where  it  was 
last  operated. 

2.  Hold  the  eqijipment  on-site  or  adjacent  to  the  point 
of  last  use. 

3.  Hold  the  equipment  in  government- c^ed  storage  sites. 

The  project  Scope  of  Work,  which  defines  the  layaway  parameters 
in  accordance  with  applicable  regulations ,  related  specifications  and 
standards,  provides  the  criteria  for  the  preparation  and  handling  of 
Industrial  Plant  Equipment  for  shipment  and  storage.    Production  equip- 
ment must  be  thoroughly  cleaned  and  preserved  to  assvoce  capability  when 
required.    Upon  project  and  program  approval,  funds  are  provided  the  PCO 
for  negotiatiOTi  with  the  contractor  concerned  for  execution  of  the  work. 
This  contract  may  be  wit±i  the  mobilization  planned  producer  or  indepen- 
dent contractor  on  an  individual  basis.    Projects  are  monitored  for 
ccxipliance  with  specifications,  quality  of  work,  and  costs,  by  the 
responsible  ARRCOM  elenents  and  assigned  DGAS  personnel. 

In  conjunction  with  the  aforementioned  layaway  procedures,  the  latest 
edition  of  Army  Regulation  700-90,  effective  15  September  75,  requires 
inspection  and  testing.    If  the  inspection  and  testing  indicates 
unsatisfactory  performance,  one  of  three  actions  will  be  taken: 

1.  The  equipment  will  be  replaced  with  equipment  frcoi 
unassigned  reserves  or  excess  inventories. 

2 .  The  equipment  will  be  repaired  or  rebuilt  to  the  extent 
necessary  to  assure  satisfactory  operation. 

3 .  The  equipment  may  be  laid  away  with  deficiencies ,  but 
in  sudh  cases,  the  deficiencies  will  be  identified  and 
recorded.    Follow-up  plans  will  be  taken  to  replace  or 
rehabilitate  the  unserviceable  equipment. 

Appendix  B  of  the  Armed  Service  Procurement  Regulation  (ASPR) 
establishes  the  requirement  that  contractors  provided  Govemment-Qmed 
property,  develop  a  plan  for  its  proper  care  and  maintenance.    This  plan 
is  subject  to  review  and  approval  by  the  Government.    Standards  for 
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equipmait  maintenance  are  normally  contained  in  either  the  supply  or 
facility  contract.    Program  authority  for  maintenance  of  equipment  in 
active  use  is  normally  provided  in  the  cost  of  the  end  item  of  the  supply 
contract  and  is  the  user's  responsibility.    Some  contracts  define  abnor- 
mal maintenance  as  a  joint  responsibility  of  the  contractor  and  the 
Government.    In  these  instances,  the  Facility  Contract  is  the  vehicle  to 
reimburse  the  contractor  for  the  Government  share  of  the  cost.  Program 
authority  for  maintenance  of  inactive  PEP's  is  provided  in  the  Government 
Facilities  Contract.    Contracts  for  maintenance  of  inactive  equipment  are 
for  a  period  of  one  year  with  option  for  renewal  for  four  additional 
years. 


SUMMARY  AND  OONCLUSICNS 

The  preceding  has  briefly  sunmarized  requirements  generation 
policies  and  procedures  relating  to  layaway,  equipment  management  and 
maintenance.    Although  the  policies  are  normally  adequate  they  cannot 
always  be  followed  precisely  due  to  limitations  in  time,  manpower  or 
funding.    In  addition,  funding  for  maintenance  of  buildings  and  equip- 
ment after  layaway  has  often  been  limited.    Lack  of  maintenance  accel- 
erates deterioration  and  decreases  readiness.    Recent  policies  and  plans 
are  directed  towards  more  reliability  and  shorter  reactivation  lead  times. 
Although  more  sophisticated  equipment  is  being  incliided  in  modernized 
lines,  information  for  layaway  and  reactivation  of  this  equipment  is 
inadequate  to  date.    In  addition,  a  more  advanced  degree  of  sophistication 
is  often  not  compatible  with  the  goal  of  rapid  reactivation. 

It  is  not  always  possible  to  attain  the  required  response  capability. 
Constant  review  and  analysis  of  contractor  capacity  and  readiness  is 
necessary  to  assess  and  determine  measures  to  inprove  the  responsiveness 
of  the  productiai  base.    In  many  cases  additional  equipment,  critical 
cooponents  or  materials  must  be  furnished  before  facilities  can  meet 
assigned  mobilization  schedules.    Projects  must  be  initiated  to  provide 
the  needed  capacity  for  new  raumitions .    In  times  of  severely  limited 
funding,  such  as  tlie  present,  the  problem  is  ccrapomded  by  the  need  to 
prioritize  and  spread  the  limited  government  resources  where  the  need  is 
most  urgent. 
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lAYAWAY,  STANDBY  AND  REACTIVATICN 


PROCEDURES  FOR  (XMPUIER  MAGNETIC  MEDIA 


INTRODUCTION 

This  paper  outlines  the  practical  procedures  which  should  be 
follcwed  in  order  to  assure  successful  preparation,  storage,  and  recall 
of  magnetic  cocputer  tapes  and  the  stored  data  vzhich  are  used  in  a  de- 
I  activated  ccnputer  ccntrolled  manufacturing  plant  which  is  to  be  re- 
I  activated  at  seme  later  time.    It  discusses  preccnditioning  and  selection 
]  procedures  for  the  layaway  phase,  the  static  and  dynamic  factors  of  the 
standby  phase,  and  finally  some  reactivation  procedures  v/hich  are  avail- 
able  for  use  with  a  magnetic  cocnputer  tape  storage  mediun. 

At  the  present  time  and  for  the  foreseeable  future,  it  appears  that 
I  magnetic  recording  will  ranain  the  principal  technological  base  for  long- 
I  term  auxiliary  storage  in  digital  conputer  systems.    There  is  no  other 

practical  approach  to  machine  readable  archival  data  storage  which  is 
I  eccncmically  superior  to  computer  magnetic  tape  recording.    For  exanple, 
conparing  the  figure  of  merit,  CxDxS  (which  is  the  information  capacity 
times  the  information  density  times  the  data  speed)  ,  of  conputer  magnetic 
tapes  to  that  of  punched  paper  cards  and  paper  tapes,  it  is  found  that 
j  cocputer  magnetic  tapes  are  superior  by  five  to  six  orders  of  magnitude. 
I  A  single  731m  (2400  ft)  long  reel  of  12. Torn  (0.5  in)  wide  magnetic 
j   computer  tape  can  store  as  much  information  as  40,000  to  200,000  -  80 
coluiiii  punched  paper  cards.    In  addition,  there  is  no  unique  environmen- 
tal advantage    to  be  gained  by  using  paper  based  storage  media  rather 
than  the  prevalent  polyester  (mylar)  based  computer  magnetic  tapes  since 
both  of  these  media  are  sensitive  to  temperature  and  humidity  conditions 
in  storage.    Therefore,  although  punched  paper  tapes  and  punched  cards 
have  been  used  in  the  past  for  auxiliary  data  storage,  only  the  layaway, 
standby,  and  reactivation  procedures  for  use  with  conputer  magnetic  tapes 
will  be  discussed  in  this  paper. 

Magnetic  recording  i-s  essentially  a  tape  surface  area  technique. 
During  the  layaway  phase,  a  recording  head  magnetizes  the  magnetic 
material  in  the  region  next  to  the  head  gap  as  a  function  of  the  input 
data.    During  the  reactivation  phase,  these  magnetized  surface  areas  are 
made  to  induce  voltages  into  a  reading  head  which  reproduces  the  stored 
digital  data  that  was  originally  encoded  magnetically  by  the  write  head. 
Ihe  principal  objectives  of  the  layaway,  standby,  and  reactivation  phases 
are  to  maximize  the  probability  of  data  recovery  by  reducing  the  poten- 
tial for  physical  damage  or  distortion  of  these  magnetized  tape  surface 
areas  and  by  recording  the  data  on  these  areas  with  techniques  vMch  are 
most  resistant  to  information  loss. 

Esqjerimental  evidence  has  shc^  that  failures  to  extract  the 
information  from  magnetic  media  are  almost  always  attributable  to  the 
physical  deterioration  of  the  media  rather  than  to  the  deterioration 
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of  the  data.    In  fact,  the  theoretical  lifetime  of  the  magnetically- 
encoded  data  is  virtually  endless  under  ideal  envirormental  and  handling 
conditions . 


LAYimY  PHASE 

There  are  three  areas  to  be  considered  in  the  layaway  process.  Ihey 
are:    the  quality  of  the  tapes  that  are  chosen;  the  physical  preparation 
of  the  tapes;  and  the  preparation  of  the  tape  contents. 


THE  QUALITY  OF  ™  TAPES  THAT  ARE  CHOSEN 

The  candidate  tapes  should  be  chosen  to  be  in  conpliance  with 
specifications  vMch  are  described  in  documents  such  as  the  U.S.  Interim 
Federal  Specif ication  WT0051C  (GSA-FSS) ,  March  1,  1975  for  12.7iiin 
(0.5in)  computer  magnetic  tapes.    Compliance  with  these  specifications 
assures  the  interchangeability  of  the  conputer  tapes  when  used  both 
internal  to  or  external  to  the  originating  cctrputer  control  system.  Each 
tape  will  not  be  restricted  to  operate  oily  with  specific  transports  and 
electronic  setups. 


THE  PHYSICAL  PREPARATION  OF  TOE  TAPES 

The  chosen  tapes  should  be  "certified,"  within  six  months  before 
being  stored,  at  the  informatiai  density  (bits  per  inch)  at  vMch  they 
will  finally  be  recorded.    The  certification  process  measures  the  signal 
anplitudes  of  pulse  trains  written  en  each  of  the  conputer  tape  tracks 
and  cotnpares  these  anplitudes  against  a  pre-established  reference  level. 
It  then  rejects  those  tapes  whose  levels  drop  below  a  reference  anplitude 
if  the  cause  for  the  si^ial  decrease  cannot  be  eliminated  using  mechanical 
or  chemical  means.    The  reference  levels  can  be  traced  back  to  the 
National  Bureau  of  Standards  Secondary  Standard  Magnetic  Tape  (Computer 
Anplitude  Reference)  SRM  3200.    It  is  recoinnended  that  a  maximum  of  five 
write  skips  should  be  allowed  for  any  chosen  conputer  tape  and  the  tape 
should  also  have  been  run  on  a  transport  for  less  than  fifty  but  more  than 
four  passes  over  its  entire  length.    Corputer  magnetic  tapes  v^ch  were 
produced  in  the  mid-1960's  or  earlier  should  not  be  chosen  even  if  they 
have  never  been  used  previously  since  it  has  been  found  that  they  have 
a  greater  tendency  to  develop  permanent  defects  as  a  functicti  of  time  in 
storage  than  tapes  which  were  developed  and  produced  in  the  1970 's. 
These  defects  develop  in  all  tapes  due  to  the  "cold  flew"  (viscoelastic) 
characteristics  of  the  plastic  constituents  of  the  tape  such  as  the 
polyester  (nylar)  base  material.    This  cold  flew  causes  physical  deforma- 
tion of  the  tape  and  is  accelerated  by  nonuniform  and  excess  stress 
buildup  in  the  tape  reel  when  it  is  subject  to  changes  in  temperature  and 
humidity.    It  has  also  been  found  that  the  biiildup  and  accumulation  of 
tape  errors  in  storage  increases  approximately  as  an  exponential  function 
of  time  and  that  errors  develop  even  in  good  storage  environments. 
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THE  PREPARATION  OF  im  TAPE  CONTENTS 

Once  the  cccputer  tapes  have  been  chosen,  then  the  infonnaticn  is 
recorded  on  them  in  the  system  format,  i.e.,  using  a  specific  bit  and 
track  density  and  modulation  method.    The  recording  system  and  transports 
should  be  carefully  adjijisted  to  minimize  head  misalignment  and  skew 
effects.    Various  investigators  have  reported  that  ccsiputer  tapes  which 
had  been  recorded  at  800  bits  per  inch  (BPI)  using  the  ncn-retum-to-zero 
(NRZI)  mode  of  recording  tended  to  develop  more  errors  in  storage  than 
tapes  \vhich  were  recorded  using  phase  encoded  (PE)  methods .  *    It  has  been 
recocinended  that  either  a  1600  bpi  PE  method  or  an  NRZI  method  at  a  lower 
information  density  such  as  200  bpi  be  used  for  archival  storage  purposes. 
Although  relatively  new,  it  appears  that  the  state-of-the-art  high  density, 
6250  bpi  ccmputer  tapes  will  probably  have  good  archival  qualities 
because  of  the  extensive  error  detection  and  correction  codes  into  which 
the  recorded  information  is  embedded.    Another  experimental  finding  of 
interest  by  NASA-GSFC  is  that  there  are  regions  along  the  length  of  many 
conputer  tapes  that  appear  to  be  more  susceptible  than  other  regions  to 
developing  data  losses  while  in  storage.    On  a  t3npical  731m  (2AO0  ft) 
length  conputer  tape,  these  undesirable  regicns  are  from  the  beginning  of 
the  tape  to  the  100m  (328  ft)  point,  from  300m  (984  ft)  to  400m  (1312  ft) 
and  from  600m  (1968  ft)  to  the  end  of  the  tape. 

It  was  noted  that  if  a  recording  strategy  could  be  devised  which 
avoided  these  regions  then  there  could  be  a  significant  increase  in  the 
laseful  storage  life  of  the  data  cti  the  tape  medium.    In  this  stucfy,  a 
tape  was  rejected  if  more  than  one  permanent  error  was  detected  over  any 
300m  (984  ft)  of  the  tape.    It  was  also  felt  that  an  effective  strategy 
might  be  developed  which  was  based  upon  the  statistically  derived  observa- 
tion diat  there  was  an  optimal  block  length  which  minimizes  tape  errors 
while  realizing  efficient  packing  of  the  data.    The  block  lengths  however 
are  usually  ccns trained  by  the  system  requiranents  and  a  practical 
application  of  this  concept  may  be  difficult  to  achieve. 

After  the  tapes  have  been  finally  recorded,  the  write  enable  ring  is 
removed  in  order  to  protect  the  data  against  erasure.    Then,  a  "read  only" 
pass  is  performed  on  each  tape  to  verify  that  it  is  conpletely  error  free 
before  being  put  into  standby.    The  tapes  should  then  be  rewound  at  the 
normal  operating  speeds  using  a  constant  tension  or  progranmed  wind  of 
six  to  eight  ounces  in  order  to  avoid  large  variations  in  pressure  from 
building  up  in  the  stored  reel.    The  winding  of  the  tapes  before  storing 
is  a  very  inportant  part  of  the  layaway  procedure  since  considerable 
mechanical  damage  can  be  caused  by  nonuniform  and  excessive  stress  build- 
up in  the  windings,  particularly  if  the  anibient  storage  temperature  and 
humidity  undergo  large  variations.    Excessive  tape  winding  pressures  and 
temperature  swings  can  produce  permanent  distortions  of  the  tape  while 
very  Ic^^  winding  pressures  can  produce  layer- to- layer  slipping  and  creas- 
ing or  "cinching"  in  the  tape  pack  which  can  lead  to  serious  data  losses. 

*This  is  due  to  the  deformation  previously  described. 
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It  has  been  noted  that  tapes  \diich  are  recorded  and  read  over  only  a 
small  portion  of  their  total  lengths  display  a  greater  tendency  towards 
cinching.    In  addition  to  winding  the  tapes  so  that  they  experience  the  \ 
proper  tension,  it  is  important  to  maintain  the  tape  transports  in  good 
mechanical  conditicn  so  that  they  will  produce  smooth  tape  winding  qualitj^ 
That  is ,  the  edge  of  the  tape  pack  should  show  no  protrusion  of  individual- 
tape  layers  after  being  wound  on  the  reel  because  this  catises  tape  edge 
damage  and  data  losses.    Ihe  hub  or  center  of  the  tape  reel  is  the 
strongest  and  most  stable  member  of  the  reel,  therefore,  the  reels  should 
be  both  handled  by  and  stored  while  supported  by  the  hub  vdienever  possible! 
Operating  personnel  should  be  instructed  not  to  handle  the  tapes  by  their 
flanges  since  this  can  cause  tape  edge  damage  as  well  as  flange  damage. 
Ihe  operating  environment  should  be  kept  clean  and  in  the  case  of  long-  ■ 
term  storage  the  tapes  should  be  stored  in  clean  transparent  plastic  ^ 
canisters  which  may  in  tum  be  sealed  in  clean  plastic  bags .  Transparent 
plastic  canisters  are  desirable  becatise  the  tape  labels  can  be  read  i 
without  opening  the  units. 

It  may  often  be  necessary  to  transport  the  tapes  to  various  vault 
or  repository  locations  for  the  standby  or  storage  phase.    Containers  ' 
should  be  used  Tfth.ich  are  designed  to  resist  temperature  variations,  ' 
moisture,  dirt,  and  shock.    The  magnetic  tape  should  be  two  to  three 
inches  from  the  outside  surfaces  of  the  container.    This  will  protect  the  i 
recorded  tapes  against  external  magnetic  fields  which  could  erase  or  ' 
damage  the  tape  signal.    Ferronagnetic  ccntainers  would  afford  considerabl 
protecticn  but  tests  have  shewn  that  this  is  an  overkill  except  for  the  ' 
most  valuable  tapes.    Considerable  efforts  should  be  made  to  control  the 
logistics  of  the  medium  transfer  to  a  vault  or  other  location.  Storage 
en  route  in  extreme  cold  or  high  temperature  environments  \diich  are  some- 
times encountered  in  uncontrolled  warehouses  should  be  absolutely  avoided 
unless  these  anibients  can  be  passed  throijgh  rapidly.    It  is  recocnmended 
that  the  media  vaults  and  the  testing  facilities  be  in  proximity  to  the 
plants.    If  a  sufficient  number  of  tiiese  deactivated  plants  exist,  it  may 
be  feasible  to  construct  centrally  located  vault  systems  and  testing 
facilities  to  service  the  entire  U.S.  ccnplex. 


STANDBY  PHASE 

Standby  is  the  program  phase  which  deals  with  the  archival  prop- 
erties of  the  media  and  system.    The  Chief  Archivist  of  the  United  States 
has  established  that  "archival"  is  defined  as  "permanent- forever."  Hcweve 
since  most  practical  stored  materials  neither  require  nor  possess  such 
extreme  capabilities,  several  other  varied  classifications  have  arisen. 
These  are:    mediim  term—up  to  approximately  10  years;  long  term- 
approximately  100  years ;  and  expendable — used  and  discarded  within  a 
short  time.    Since  it  has  been  estimated  that  present  day  conpjter 
magnetic  tapes  have  a  shelf  life  of  approximately  15  to  20  years,  it 
appears  that  the  standby  phase  should  be  considered  as  a  medium  term 
archival  program.    It  will  be  seen  that  the  dynamic  portion  of  the  standby 
program  can  extend  the  archival  time  well  beyond  the  mediun  term. 
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Itie  standby  program  consists  of  a  static  phase  and  a  dynamic  phase. 
.  The.  static  phase  deals  with  the  control  of  the  ambient  storage  environment 
I  and  the  physical  hoijsing  of  the  magnetic  media.    The  dynamic  or  active 

portion  of  the  standby  program  consists  of  measures  which  can  be  undertaken, 
■  often  on  a  scheduled  basis,  to  prevent,  to  seek  out,  and  to  circumvent 

tiie  loss  of  stored  data  due  to  the  physical  distortion  of  the  medium. 

)  STATIC  PHASE 

j  The  static  measures  consist  of  maintaining  a  proper  temperature, 

humidity,  and  security  envircnment  for  the  stored  t^es,    A  good  storage 
environment  is  one  which  approaches  the  characteristics  of  a  tape  test 
laboratory.    Although  the  very  high  class  clean  room  conditions  which  are 
produced  for  exariple  with  laminar  air  flow  systems  through  electrostatic 
j  filters  are  not  needed,  positive  internal  air  pressure  systems  in  the 
I  vaults  are  highly  recoimiended.    This  will  prevent  the  intrusion  of  external 
'  dust  into  the  storage  area.    The  vault  temperature  should  be  maintained 
at  approximately  15.6  C  (60  F)  to  26.7°C  (80°F)  at  a  relative  humidity 
range  of  from  407o  to  60%,    Whenever  possible,  the  tenperature  and  humidity 
of  the  media  vault  should  be  close  to  that  of  the  operating  ccnqxiter 
environment  in  order  to  avoid  subjecting  the  tapes  to  large  ambient  changes 

they  are  transferred  from  the  vault  to  the  cconputer  installaticn. 
The  tapes  should  always  be  stored  in  an  ijpright  position  and  supported 
by  their  hubs  in  the  enclosed  canister. 

There  are  comnercial  organizations*  available  that  provide  secure 
storage  vault  services  for  ccmputer  magnetic  tapes,  microfilms,  and 
paper.    They  are  often  located  underground  in  order  to  protect  the  stored 
materials  against  e^losion.    They  are  held  at  a  constant  tenperature 
and  humidity  and  employ  electronically  filtered  air  systems.    The use  of 
these  coomercial  repositories  would  be  very  valuable  if  a  redundant  storage 
approach  is  taken,  i.e. ,  duplicate  tapes  are  made  for  all  or  part  of  the 
tape  library  and  stored  in  various  locations  as  a  precautionary  measure. 

It  has  been  found  that  a  magnetic  field  supplies  the  only  kind  of 
energy  that  can  cause  undetected  data  destruction  without  any  accooopany- 
ing  physical  distortion  or  damage  to  the  magnetic  storage  media.    It  has 
been  determined  experimentally  that  normally  there  is  no  need  to  shield 
the  stored  data  against  x-rays,  high  voltage  fields,  nuclear  radiation, 
high  frequency  fields,  or  light  energy.    Most  inportant,  a  spacing  of 
only  a  few  inches  is  sufficient  to  protect  the  recorded  media  against 
magnetic  fields  \diich  are  far  more  intense  than  are  ever  found  in  the 
normal  environment  or  that  can  be  produced  by  a  concealable  permanent 
magnet. 

*For  example,  see  Information  and  Records  Management,  page  38,  January  1977. 
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Finally,  security  measures  should  be  undertaken  to  protect  the  tape 
vault  against  unwarranted  intrusion  or  catastrophic  damage.    This  is  done 
with  monitoring  devices  such  as  magnetometers  and  properly  insulated 
vault  construction  v^ch  can  protect  the  tapes  against  fire  hazards. 
Fire  has  been  found  to  be  the  greatest  threat  to  the  magnetic  tape 
repository. 


DYNAMIC  PHASE 

The  dynamic  porticn  of  the  standby  program  consists  of  ongoing 
measures  vMch  can  be  instituted  to  iaprove  the  chance  of  survival  of 
stored  magnetic  cco^juter  tapes  and  their  cctitents.    These  measures  include 
tape  rewinding,  physical  inspection,  recopying  of  key  tapes,  and  sched- 
uled reading  of  sanple  tapes  from  all  of  the  tape  files. 

It  has  been  found  that  if  the  tapes  are  "exercised"  by  winding  and 
rewinding  them  several  times  on  an  annual  or  semiannual  basis  before  tape 
errors  have  formed  that  many  errors  can  be  circumvented.    The  rewinding 
process  which  is  performed  at  nonnal  operating  tape  speeds  tends  to 
remove  sane  of  the  stresses  which  have  built  up  in  the  tape  reel  during 
the  storage  interval  and  can  remove  incipient  errors. 

There  are  two  types  of  tape  errors:    tenporary  and  permanent. 
Tenporary  errors  can  usually  be  removed  by  cleaning  and  rewinding  the 
tape,  v^le  permanent  errors,  v^ch  are  caused  by  defects  such  as  missing 
oxides,  deep  scratches  and  creases,  are  not  removable.    It  has  been  found 
that  even  tonporary  tape  errors  tend  to  become  pennanent  if  they  exist  for 
a  long  time  in  storage.    Usually  a  small  sample  (apprcodmately  5%)  drawn 
frcm  a  large  tape  file  is  representative  of  the  tapes  in  that  same  file. 
It  is  recommended  that  a  randan  sanple  of  tapes  fran  each  file  should  be 
read  on  an  annual  basis  and  checked  for  permanent  errors.    If  these  errors 
exist,  it  may  be  necessary  to  read  through  the  entire  file  and  recopy  or 
replace  all  of  the  defective  tapes.    Ccooputer  installation  managers  will 
scmetimes  recopy  their  inportant  tapes  on  an  annual  basis  as  a  normal 
routine.    If  there  are  no  permanent  errors  on  a  stored  computer  tape, 
then  it  is  possible  to  recopy  the  data  onto  a  new  tape  and  to  rehabilitate 
the  original  tape  for  future  use.    Rehabilitation  of  good  used  tapes 
consists  of  erasing,  cleaning  (wet  or  dry)  ,  recertifying,  rewinding,  and 
relaxing  the  tapes  before  returning  them  to  service.    Ihere  are  comnercial 
organizations  vMch  specialize  in  tape  rehabilitation.    Some  Government 
organizations  such  as  NASA  have  an  in-house  tape  rehabilitation  facility. 

Tapes  should  also  be  inspected  visually  for  any  physical  defects 
such  as  cinched,  i.e.,  creased  and  folded  over  layers  of  tape,  warped 
tape  edges,  discoloration,  etc.    Tapes  which  display  these  defects 
should  be  checked  iinnediately  for  data  losses,  recopied,  and  replaced 
if  necessary. 
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REACTIVATION  PHASE 


Hie  success  of  the  reactivation  phase  is  strongly  influenced  by  the 
measures  that  are  taken  in  the  layaway  and  standby  operations. 

When  the  conputer  tapes  are  brought  out  to  the  coiputer  installation 
from  the  vault,  they  should  be  permitted  to  adjust  themselves  to  the 
anfoient  tenperature  and  humidity  of  that  environment  for  approximately 
24  hours  before  being  placed  into  operation.    The  tape  operating  envi- 
rcranent  at  the  cooputer  installaticn  should  also  be  maintained  as  a  clean 
area  by  using  positive  internal  air  pressures  at  flows  of  several  thousand 
CEM. 

Avoid  bringing  products  such  as  paper  stock  and  cardboard  or  other 
lint  producing  items  into  the  high  pressure  section  of  the  installaticn 
where  the  tape  transports  should  be  located.    No  cigarette  smoking  or 
eating  of  food  should  be  permitted  in  the  conputer  room.    The  operators 
should  be  trained  to  handle  the  reels  by  their  hubs  and  not  by  their 
flanges  and  should  return  the  tapes  to  their  canisters  vThen  not  being  used. 
The  tape  operators  should  use  clean  lint- free  smocks  (for  example,  100% 
dacron  polyester)  and  gloves  vMle  on  duty. 

The  recovery  of  the  original  data  from  the  stored  tapes  is  the  in- 
dicator of  success  for  the  entire  operation.    During  the  reactivation 
phase,  the  data  base  vdiich  has  been  stored  on  the  computer  tape  is  trans- 
ferred into  the  computer  to  load  the  system  or  it  is  transferred  to 
peripheral  devices  such  as  rapid  access  magnetic  disks  for  cn-line  opera- 
tions.   In  the  event  that  data  loss  problems  are  found  to  have  developed 
during  this  transfer  phase,  there  are  some  corrective  measijres  which  can 
be  mdertaken.    For  exanple,  it  was  noted  previously  that  tapes  T^ich  are 
recorded  at  800  bpi  in  the  NRZI  mode  tend  to  lose  data  easily.    The  cold 
flow  distorticn  and  sksdng  of  the  tape  is  believed  to  be  the  principal 
caxose  for  the  data  loss  at  this  density.    Sometimes  the  tape  can  be 
sufficiently  straightened  by  rewinding  it  at  normal  speeds  several  times, 
and  then  relaxing  the  tape  for  one  or  two  days  and  then  finally  winding 
and  rewinding  it  a  nuniber  of  times  on  the  transport  which  will  be  used  to 
finally  read  the  tape.    This  same  procedure  can  be  initiated  before  read- 
ing a  tape  if  it  is  noted  that  the  tape  windings  appear  to  be  irregular 
with  sane  tape  edges  protruding  from  the  pack. 

There  is  a  wide  range  of  electronic  and  mechanical  transport  adjust- 
ments vMch  can  often  be  performed  by  technical  personnel  which  will 
recover  marginal  tape  information.    Quite  often  a  tape  which  is  difficult 
to  read  on  one  transport  will  be  read  easily  on  another.    It  has  been 
noted  that  seme  tape  drives  which  have  the  capability  of  reading  backwards 
can  often  read  a  balky  cccoputer  tape  \^ch  resisted  being  read  in  the 
forward  direction. 
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Measijres  have  been  described  which  can  be  applied  during  the  layaway 
and  standby  phases  of  the  cccputer  magnetic  tape  program;  these  measures 
greatly  enhance  the  probability  of  successful  reactivation  and  final  data 
recovery.    It  should  be  noted  that  this  paper  has  outlined  only  a  portion 
of  the  possible  actions  which  can  be  taken  to  preserve  the  media  and  its 
data  and  that  there  is  a  considerable  amount  of  research  vMch  must  still 
be  done  in  the  area  of  the  archival  properties  of  magnetic  media.  The 
guidelines  which  have  been  presented  in  this  paper  are  generally  applicable 
to  all  flexible  magnetic  storage  media  with  some  additions  required  for 
different  media  housings.    It  is  not  anticipated  that  large  data  base 
archival  systems  will  use  coiputer  magnetic  disk  pack  systems  in  lieu  of 
the  lower  cost  magnetic  tape  systems. 

The  following  is  a  sunmary  of  the  practical  procedures  which  have 
been  described  in  the  paper.    They  assure  the  successful  reactivation  of 
magnetic  ccoputer  tapes  and  the  recorded  data. 


lAYAMAY  PROCEDURES 

1.  Purchase  computer  magnetic  tapes  which  are  based  upon 
specifications  such  as  U.S.  Interim  Federal  Specification 
Wr-0051C  (GSA-FSS),  March  1,  1975. 

2.  The  magnetic  tapes  should  be  certified  within  six  months 
prior  to  being  stored. 

3.  Permit  a  maximum  of  five  write  skips  cn  any  chosen  magnetic 
tape.  The  tapes  should  have  been  run  less  than  50  but  more 
than  4  full  passes  before  testing  or  storing. 

4.  Purchase  tapes  which  have  been  produced  within  the  last 
five  years. 

5.  Avoid  the  800  NRZI  data  recording  format. 

6.  Perform  a  "read-only"  tape  pass  prior  to  storage. 

7.  Reswind  the  tapes  at  normal  speeds  using  constant  tension 
or  programned  winding  techniques . 

8.  Package  the  magnetic  tapes  in  clear,  transparent  canisters. 

9.  Shipping  containers  should  resist  temperature  variations, 
moisture,  dirt,  and  shock. 

10.    Prolonged  tenperature  and  humidity  extremes  should  be 
avoided  ^en  the  tapes  are  transported. 
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STANDBY  PROCEDURES  (STATIC) 

1.  Maintain  vault  taiperatures    ranges  of  approximately 
IS.e'^C  (60°F)  to  26.7  C  (80°F) . 

2.  Maintain  a  vault  humidity  range  of  from  40%  to  60% . 

3.  Enploy  positive  internal  air  pressure  systems  to  reduce 
dust  in  the  vault. 

4.  Use  cctimercial  vault  systems  for  redundant  tape  storage 
if  feasible. 

5.  Store  tapes  in  an  \:^right  (vertical)  position. 

6.  Maintain  vault  security  against  entry  by  unauthorized 
personnel. 

7.  Use  magnetcmeter  detection  for  concealed  permanent  magnets 

8.  Maintain  insulated  vaults  against  fire  hazards.  Insulate 
the  vaults  for  fire  resistance  up  to  at  least  66  C  (150  F) 
Allcw  a  minimum  of  ccndbustible  niaterials  within  the  vault 
area. 

9.  Carbon  dioxide  (CO2)  and  water  are  permissible  for  fire 
extinguishing. 


STANDBY  PROCEDURES  (DYNAMIC) 

1.  Exercise  each  stored  tape  on  an  annual  or  semiannual 
scheduled  basis. 

2.  Read  at  least  one  tape  fron  each  small  tape  file  annually 
and  a  5%  random  sample  frcm  each  large  tape  file.  Recopy 
and  replace  defective  tapes  which  have  shown  data  losses. 

3.  As  an  option,  recopy  very  inportant  tapes  oa  an  annual 
basis. 

4.  Inspect  tapes  for  creases,  warped  edges,  discoloration, 
or  other  physical  ancmolies.    Recopy  and  replace  if 
necessary. 

5.  Maintain  all  of  the  operating  magnetic  tape  transports 
in  a  clean  and  properly  adjusted  condition.    Check  for 
points  of  wear  in  the  tape  head  and  guidance  system 
vMch  can  cause  tape  damage. 
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REACTIVATION  PROCEDURES 

1.  Relax  the  tapes  in  the  ccraputer  environment  for  24 
hours  before  they  are  run. 

2.  The  coEcputer  installation  should  be  maintained  as  a 
clean  area  by  using  positive  internal  air  pressures. 
Avoid  introducing  dust  or  lint  producing  products  into 
the  ccoputer  area. 

3.  Operators  should  be  trained  relative  to  the  proper 
handling  of  the  magnetic  cccoputer  tapes. 

4.  Exercise  tapes  which  display  skew  losses  or  \^;hich  are 
not  evenly  wound. 

5.  Technical  personnel  can  recover  marginal  data  through 
electronic  and  mechanical  adjustments. 

6.  Balky  conputer  tapes  can  sometimes  be  read  in  reverse 
more  effectively  then  in  the  forward  direction. 


The  information  and  reccranendaticns  in  this  paper  have  been 
asserrbled  from  various  sources  including  personal  investigations  and 
private  conmi  mica ticns ,  particularly  with  moribers  of  the  National 
Archives  and  Records  Services  staff  to  whom  the  author  is  indebted. 
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ENVHOT'ENrAL  EFFECTS  ON  ELECTRICAL  AM)  ELECTRONIC 
DEVICES  AND  EQUIPMEtTT  UNDER  IJDNG  STORAGE  CONDITIONS 


An  important  aspect  of  the  Base  Modernization  Program  (under  the 
direction  of  the  Project  Manager's  Office  for  Munition  Production,  Base 
Modernization  and  Expansion)  is  to  take  advantage  of  historical  informa- 
tion and  modem  expertise,  and  to  insure  that  facilities  which  produce 
munition  type  items  can  be  mobilized  quickly  and  effectively.  Automtic 
processing  has  required  sophisticated  computer  control  systems  to  perform 
a  variety  of  functions  with  reliability  and  speed,  hence  reducing  human 
error  v^le  increasing  safety  and  production  rates.    It  has  been  shown 
in  a  broad  range  of  systens  and  industries  that  ccmputer  control  produc- 
tion lines  can  be  extremely  effective,  both  cost  and  reliability  wise. 
Although  the  sane  advantages  apply  to  Munition  production,  a  distinct 
difference  exists:    the  base  modernization  program  requires  shutting  down 
for  long  periods  of  time,  referred  to  basically  as  layaway.  After 
periods  of  dormancy  there  will  be  sane  effect  on  the  ability  to  restart 
the  syston  properly,  impeding  needed  mobilization  rates,  effectiveness  and 
safety.    The  degree  of  these  problems  is  quite  dependent  on  the  attention 
we  pay  to  shut  down,  layaway  and  restart  procedures  and  problen  identi- 
fication to  address  dormancy  effects. 

This  paper  will  address  the  control  system  and  the  interface  elec- 
tronics.   It  is  well  recognized  that  other  problems  are  associated  with 
layaway  v\^ch  are  not  to  be  minimized.    However  since  our  interests, 
experience  and  expertise  lie  basically  in  understanding  failure  mechanisms 
and  failure  modes  of  microelectronic  devices,  this  is  our  present  area 
of  ooncem. 

This  discussion  will  be  divided  into  two  phases .    First  is  a  dedi- 
cated effort  in  the  area  of  failure  analysis.    Second  is  a  standby 
monitoring  of  subsystems  to  observe  inportant  parameter  values  for 
determining  how  degradation  proceeds  to  adequately  signal  v;hen  out  of 
spec  or  catostrophic  failures  occur. 

A  discussion  of  these  two  phases  follows  but  first  a  basic  prenise 
should  be  considered.    A  most  obvious  question,  "Does  time  alone  degrade 
microelectronic  devices?"    That  is,  if  a  reliable  system  is  operating 
in  a  prescribed  fashion,  and  left  in  a  dormant  state  (even  with  the  ton- 
perature  and  humidity  control)  will  the  system  degrade  as  a  function  of 
time? 

Although  information  in  this  area  is  limited,  it  appears  the  answer 
to  this  question  is  a  definite  yes.  This  is  based  on  observation  in  the 
technical  camunity  and  in  particular  the  munitions  industry.  Stockpile 
testing  show  that  after  a  syston,  either  cctiplex  or  sinple,  has  been 
left  in  a  dormant  environment  and  then  put  through  tlie  same  testing  pro- 
gram as  similar  units  had  passed  earlier,  a  marked  change  in  reliability 
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mechanism  and  its  trend  before  catostrophic  failures  occur.    This  is  one 
single  situation  indicative  of  the  role  and  value  of  the  failure  analysis 
facility  for  establishing  an  effective  layaway  program. 

An  even  better  way  to  address  the  above  problem  can  be  acccmplished 
by  placing  components  in  a  laboratory  storage  test  and  evaluation  program 
simultaneous  with  their  life  cycle  in  the  plant.    Therefore,  let's  look^ 
at  a  unique  cctipleraentary  way  to  track  and  prevent  failures  from  occurring 
in  our  systems  and  nore  accurately  and  timely  determine  how  the  dormant 
environment  affects  microelectronic  devices  of  concern. 

To  accctrplish  this  we  are  proposing  test  and  evaluation  of  selected 
conponents  to  be  done  on  a  coninuing  basis  using  the  Long  Term  Dormant^ 
Storage  Data  Acquisition  Facility,    We  are  suggesting  to  withdraw  specific 
spare  parts  normally  obtained  frcm  stock  kept  at  the  site.    These  may 
include  a  variety  of  items  including  sensors.    A  selection  of  these 
conponents/ sub-systems  will  be  made  and  used  as  prime  candidates  to  be 
maintained  in  a  simulated  storage  environment  in  the  laboratory  at  the 
ARRADCOM  facility  and  specific  electronic  monitoring  of  these  items  will 
be  performed.  Test  sets  and  systems  software  would  be  developed  to  auto- 
matically measure  inportant  parameters  on  a  routine  basis  for  permanent 
ccnputer  storage.    The  accumulated  data  will  also  be  analyzed  routinely. 
Small  changes  in  tlie  operating  parameters  of  the  components  on  the  board 
will  be  observed  and  as  realizable  the  trends  of  degradation  which  occur 
in  these  microelectronic  devices  will  be  gleaned   and  the  mechanisms  vM.ch 
are  causing  these  trends  to  occur  will  be  evaluated.    Determination  of 
how  these  trends  vjill  effect  efficient,  reliable  and  safe  operation  will 
be  established  and  recomiendation  to  AE^COM  will  establish  what  type  of 
action  is  needed  to  either  curtail  the  trends  or  to  engineer  your  way 
around  them.    Feedback  to  the  periodic  maintenance  procedures  is  the  most 
likely  way  to  keep  the  control  system  operational.    This  is  also  the  most 
effective  way  to  prevent  catostrophic  failures  from  occuning  vMch  may 
occur  at  a  time  viien  system  operation  is  required.    The  ARRADCCM  has  an 
ongoing  facility  to  measure  fuze  conponents  in  the  dormant  environment 
specifically  designed  to  determine  how  device  degradation  occurs  in 
relation  to  years  of  this  type  of  storage.    It  is  a  straightforward 
matter  to  incorporate  a  large  number  of  the  components  and  subsystems 
from  the  base  modernization  program  into  our  dedicated  Dormancy  Evalua- 
tion Facility.    Once  trends  have  been  established  and  failure  mechanians 
determined  a  greater  degree  of  assurance  (of  how  devices  will  perform  in 
a  dormant  environment)  can  be  made*  Needed  changes  could  be  prescribed 
for  our  new  systems. 

This  paper  above  describes  twD  extremely  inportant  areas  vdiich  need 
to  be  addressed.  It  makes  poor  engineering  sense  in  our  estimation,  to 
do  maintenance  on  a  system  without  benefit  of  detailed  failure  analysis 
to  determine  the  cause  of  failure.  In  any  case  this  information  is  ex- 
tremely inportant  in  future  layaway  systems  and  will  help  improve 
maintenance  of  present  systems  and  perhaps  allow  the  maintenance  inter- 
val to  be  expanded  i^:iich  would  reduce  the  cost  of  maintenance.  In 
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and  functioning  is  noticed.    Similar  observations  were  noted  by  ^fertin 
Marietta,  in  two  separate  studies  \Ad.ch.  have  shown  that  degradation  of 
microelectronic  components  does  take  place  as  a  resiiLt  of  storage  alone. 
This  w^rk  was  reported  almost  a  decade  ago.    Revisions  to  their  report 
have  been  made  in  the  last  few  years.    The  wrk  clearly  indicates  that 
microelectronic  devices  and  electronic  devices  in  general  do  degrade  as 
a  function  of  time.    Although  this  degradation  may  be  slow  ccnpared  to 
the  microelectronic  degradation  rates  xdiich  are  published  for  active 
devices,  the  degree  of  degradation  can  be  important  to  a  particular 
storage  situation.    It  appears  certain  that  time  alone  does  change 
microelectronic  component  behavior.    Since  microelectronic  cocponents  are 
used  extensively  throughout  the  base  modernization  program  a  layaway 
program  must  address  this  aspect  if  a  position  of  readiness  is  to  be 
realistic. 

It  is  assumed  that  the  plant  control  system  facilities  will  be  put 
into  some  sort  of  dormant  or  layaway  status.    For  the  purpose  of  this 
discussion  assime  that  the  control  system  and  housing  will  be  left  in  a 
tenperature/hunidity  controlled  environment,  (toiperature  vrould  be  main- 
tained at  approximately  75°  F  +  5°  in  the  humidity  will  be  controlled  to 
some  value  less  than  507o) .    At  sctne  period  of  time  (this  period  is  open 
to  discussion  and  should  be  a  major  point  of  this  conference,  however, 
will  not  be  addressed  in  this  paper)  the  entire  system  will  be  reactivated 
following  prescribed  start  up  procedures.    As  maintenance  personnel 
perform  this  task,  specific  adjijstment  and  calibrations  will  be  made  and 
the  checkout  procedures  should  identify  by  system,  subsystem,  assembly  or 
cooponent,  any  out  of  spec  operation.    Each  s\ich  item  should  be  removed 
and  replaced.    These  defective  items  should  be  identified  to  the  ARFADCXM 
failure  analysis  point  of  contact  and  returned  as  per  their  instructions. 
Given  these  assumptions  we  will  indicate  how  the  failure  analysis  group 
should  be  dedicated  to  determining  the  failure  modes  and  trends  that  exist 
in  dormancy  and  interface  with  the  layaway  program. 

Upon  receipt  at  the  central  failure  analysis  facility,  a  caiplete 
diagnostic  analysis  will  be  performed.    The  item  will  be  cataloged  for 
identification  with  details  such  as  its  lot,  date  code.    The  specific 
failure  mode  will  be  analyzed  and  the  failure  mechanism  determined.  The 
realtionship  of  the  failure  to  the  system  performance  and  safety  will  be 
determined;  e.g. ,  is  the  failure  mechanism  generic  or  randan  in  nature. 

Control  systems  of  the  type  we  are  concerned  with  generally  have  a 
number  of  repetitive  device- types  since  the  design  engineer  or  cocpany 
applies  similar  logic  elements  as  many  places  as  possible  for  design 
purposes  as  veil  as  to  reduce  inventory  and  parts  control.    This  also 
breeds  greater  familiarity  with  that  cccponent  among  the  design  engineers 
and  cormonality  of  device  selection  can  propagate  further.    "IMs  can 
give  a  distinct  disadvantage  to  the  user  if  a  defect  does  exist  in  a 
device  type  of  a  particular  lot  date  code  as  it  may  exist  in  all  of  those 
ccoponents  with  the  same  lot  date  code.    Even  worse,  it  could  exist  in 
all  of  the  devices  made  by  that  same  technology.    It  is  therefore 


inportant  to  determine  the  failure  mode  and  identify  v^ther  it  vjas  a 
random  or  other  type  of  failure  mechanism.    Failures  can  be  caused  to 
occur  due  to  improper  stimulization  in  either  the  shut  down  or  start  up 
procedure  hence  root-ca\ase  also  needs  to  be  classified  as  well. 

An  additional  advantage  of  a  central  failure  analysis  facility, 
controlling  the  detailed  failure  analysis  and  categorization,  is  the 
ability  to  maintain  an  exact  record  of  how  specific  coiponents  and 
technologies  relate  to  performance  and  safety  in  the  layaway  environ- 
ment.   Also  determinations  can  be  appropriately  feedback  to  the  design 
engineers . 

A  sinple  illustration  may  be  given:    consider  a  hypothetical  case  of 
a  plant  i^ch  has  been  successfully  checlced  out  and  tlie  control  system 
has  functioned  properly.    The  system  is  shut  dovm  following  prescribed 
guidelines  and  a  period  of  dormancy  ensues,  say  for  approximately  one 
year.    The  maintenance  organization  performs  system  start-up  following 
prescribed  procedures.    A  number  of  adjustments  were  required  in  the 
voltage  stabilization  portion  of  the  logic  section  however  tolerances 
could  be  met  successfully.    The  unit  was  returned  to  layaway  for  another 
year  and  periodic  maintenance  was  again  performed.    This  time  it  was  im- 
possible to  ccuplete  the  calibration  of  the  voltage  regulation  circuit. 
Appropriate  microcircuit  cards  were  withdrawn  and  new  ones  were  plugged 
in  allowing  calibration  to  be  carpleted.    Removed  cards  were  returned  to 
the  failure  analysis  facility.    The  failure  mode  was  analyzed  and  is 
determined  to  be  a  shift  of  the  Zener  parameter  causing  an  out  of  spec 
behavior.    The  trend  indicates  adjustments  at  the  plant  will  soon  not  be 
able  to  be  made  to  attain  the  proper  output.    Circuit  evaluation,  analy- 
sis of  test  data  (available  and  generated)  is  combined  witli  special 
examination  of  tlie  solid  state  coiponents  to  isolate  die  xener  as  the 
culprit.    The  Zener  is  then  scrutinized  with  further  special  tools  such 
as  the  scanning  electron  microscope,  microprobe,  and  optical  microscopy. 

The  .analysts  established  that  the  failure  was  due  to  oxide  breakdown 
to  a  type  defect  comnon  in  that  particular  lot  date  code.  Regulators 
frcm  other  control  boards  from  that  system  are  examined.    It  is  deter- 
mined a  trend  in  parametric  behavior  is  present  which  shows  degradation 
is  taking  place.    Effected  boards  can  be  withdrawn  frcm  the  stoclqjile 
and  retrofitted  with  a  newly  selected  diode  which  are  not  subject  to  the 
same  type  of  failure  mechanism. 

Now  assijme  failure  analysis  was  not  done  in  the  suggested  fashion 
and  maintenance  again  tries  to  adjust  tlie  system  as  prescribed.  Again 
they  would  find  tliat  they  have  to  adjust  the  voltage  regulation  circuit 
and  perhaps  one  or  two  cards  are  replaced.    The  system  continues  until 
the  inevitable  time  the  Zener  diode  fails  catastrophically  causing  the 
board  to  bum  out  or  causing  serious  voltage  stressing  of  other  compon- 
ents in  the  system.    This  is  not  a  far  fetched  hypothesis  and  in  fact 
has  been  observed  to  exist  in  similar  situations.    It  is  clear  frcxn  this 
hypothetical  case  that  you  would  much  prefer  to  determine  the  failure 
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addition,  we  have  an  opportunity  to  examine  selected  devices  in  a  con- 
trolled environment  and  determine  the  effects  of  dormancy  on  a  routine 
basis .    Hence  we  can  understand  the  behavior  of  the  devices  with  time , 
and  this  treid  infonnation  can  be  applied  to  our  present  systems  as  well 
as  those  being  developed. 
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A  lAYPm^i  PROGRAM  FOR  OQtTTROL  VALVES 


INrRODUCriON 

It  is  tile  intent  of  this  presentation  to  bring  together  in  outline 
form  many  of  the  factors  that  should  be  considered  for  a  layaway  program 
for  control  valves  with  a  view  to  stimulating  discussion  and  further 
analysis .    There  will  be  many  ramifications  and  special  situations ,  but 
I  think  wt  can  say  there  are  three  basic  steps  involved  in  the  prepara- 
tion of  a  urogram. 

First,  guidelines  and  objectives  must  be  specified. 

Next,  instructions  and  layaway  data  should  be  prepared. 

Finally,  there  must  be  a  recordkeeping  syston  for  this  data  plus 
equipment  records. 


GUIDELINES  AND  OBJECTIVES 

Ground  rules  for  the  plant  as  a  whole  presumably  will  be  available 
nhere  needed.  Some  of  the  items  to  include  in  the  ground  rules  are  the 
following: 

E}q)ected  duration  of  a  startup 

Cost  of  delays  during  startup 

Hie  degree  and  nature  of  monitoring  during  shutdown 
Minicajm  and  maximim  duration  of  the  shutdown 
^feterials  and  methods  for  recordkeeping 

Reconmended  practices  for  handling  time-related  deterioration 


INSTRUCTIONS  AND  lAYmC£  DATA 


Collecting,  screening,  and  documenting  data  for  layaway  instructions 

III  requires  engineering  consideration  and  time,  hence  it  is  costly.  Some 

!j  generalized  data  may  be  obtained  from  equipment  sijppliers ,  but  the  main 

I  burden  will  fall  on  in-house  personnel.    A  prime  and  valuable  source  of 

j  data  is  the  on-site  operating  and  engineering  staff       will  have  first- 

|i  hand  experience  with  the  various ,  and  often  noxious ,  process  fluids  en- 

i  countered. 
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The  principal  problem  for  valve  layaway  programs  is  corrosion.  If 
a  carbon  steel  valve  can  be  flxished  after  shutdown  and  left  in  a  warm, 
dry  place  with  no  pockets  of  fluid  inside,  it  should  stand  up  indefinite- 
ly.   On  the  other  hand,  moisture  pockets  in  various  areas,  such  as  the 
lower  portion  of  a  stuffing  box,  will  cause  corrosion  in  carbon  steel. 
Just  the  moisture  condensation  frcm  atmospheric  conditions  in  sunner 
will  cause  rusting  in  a  carbon  steel  valve.    But  valves  are  not  unique 
in  this  r^ard,  and  they  can  be  treated  in  the  same  manner  as  other 
carbon  steel  equipment.    It  is  also  important  to  flush  and  clean  out 
corrosive  solutions  frcm  an  alloy  steel  valve.    Keeping  a  roof  over  idle 
equipment,  and  preventing  sudden  changes  in  temperature  will  help  pre- 
vent corrosion.    For  noncritical  areas,  a  anall  amount  of  rust  on  equip- 
ment can  usually  be  tolerated.    When  rusting  becomes  a  problon,  you 
should  consider  replacing  carbon  steel  with  stainless  steel. 

Dissimilar  metals  in  contact  with  one  another,  as  you  know,  are 
siibject  to  galvanic  corrosion.    Typically  we  have  found  that  #416  stain- 
less steel  in  contact  with  nickel  will  pit  within  six  months.    Any  of 
the  #400  series  stainless  steels  in  contact  with  a  #300  stainless  steel 
can  be  a  source  of  galvanic  corrosion.    Particular  itans  to  watch  are 
valve  plugs  and  plug  seals  in  contact  with  valve  guides.    Another  item 
is  valve  stons  in  contact  with  stem  bushings.    Stellite  6  and  other  co- 
balt alloys  are  almost  entirely  free  from  galvanic  corrosion  and  are 
good  materials  for  these  applications .    Teflon  coatings  or  parts  are 
equally  corrosion  free.    It  is  easy  to  concltide  frcm  these  observations 
that  specification  of  the  proper  materials  in  the  early  desig;n  stages 
can  reduce  cleaning  and  storing  problans  at  layaway  time. 

When  two  surfaces  are  in  contact  with  each  other  and  subject  to 
vibration,  fretting  may  occur.    Normally  two  surfaces  stored  in  contact 
will  be  in  a  static  condition,  particularly  when  an  operation  is  shut 
down.    However,  fans  or  conpressors  in  operation  nearby  can  create 
enough  vibration  at  times  to  be  a  hazard  to  stored  equipment. 

Elastomers  have  a  limited  shelf  life.    Elastomer  rings  for  slipper 
seals,  seat  ring  seals,  actuator  diaphragns,  and  static  "0"  ring  seals 
are  a  few  of  the  itens  in  this  category.    When  items  are  made  of  elasto- 
mer materials,  their  shelf  life  is  a  factor  governing  a  layaway  program. 

As  you  know,  teflon  has  a  tendency  to  flow  under  pressure.    It  is 
essential  to  reckon  with  this  factor  ^^hen  planning  the  stock  for  re- 
placements of  teflon  itans  such  as  soft  seats  for  valves. 

Any  plants  located  near  salt  water  will  have  a  special  problem  with 
chlorides.    All  steels,  even  #316  stainless  steel,  are  subject  to  stress 
corrosion  vhen  in  contact  with  chloride  solutions.    Potential  damage 
frcm  this  factor  should  be  assessed  and  all  reasonable  actions  taken  to 
limit  the  corrosive  effects.    When  the  chlorides  are  present  in  the 
process,  one  would  expect  to  find  extensive  use  of  alloys  such  as  >SDnel, 
Hastelloy  C,  and  Hastelloy  B. 
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Sometimes  it  is  necessary  to  take  certain  valves  apart  at  shutdown 
to  clean  them  or  dry  out  pockets  of  fluid.    When  this  is  done,  the  valve 
should  be  serviced  and  stored  ready  for  startup.    Usually  carbon  steel 
valves  can  be  stored  in  the  pipeline.    It  is  desirable  to  monitor  the 
valve  condition  periodically  on  representative  units  and  record  findings. 

Valve  positioners  and  signal  converters  should  be  recognized  as 
instruments  and  treated  accordingly.    Even  though  they  may  be  assonbled 
as  an  integral  part  of  the  actuator,  the  v^hole  asserribly  should  be 
treated  as  an  instrument.    This  may  required  indoctrination  of  personneL 

The  considerations  discussed  are  representative  of  many  operating 
conditions  found  in  plants  and  processes.  The  list  is  almost  endless, 
and  I  invite  you  to  add  to  it  from  your  experience. 


j  DOCUMENIATION 

'         Records  are  essential  so  that  guidance  and  basic  information  may  be 
j  available  to  those        follow.    The  subject  can  be  divided  into  five 
main  topics: 

!         Control  valve  records 

!         Specification  and  procurement  instructions 

Shutdown  procedures  (reference  data) 

I         Layaway  procedures  (reference  data) 

I         Startup  procedures  (reference  data) 

^^ost  of  these  topics  are  interrelated  and  can  seldom  be  analyzed 
1  independently  of  one  another. 

Generally  the  valve  layaway  records  should  form  a  part  of  the  equip- 
!  ment  and  maintenance  records .    Control  valve  records  could  look  like 
!!  this: 

Control  Valve  Records 

i  a.    Identity  -  tag  and  location  in  process  line 

b.    Procuranent  data 

iSi^plier 
Manufacturer 
Order  No. 
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Date  Purchased 
Serial  nuinber 
Size 

Body  material 
Class 

End  connections 
Actuator  pressures 
Bench  Open 
Process  Open 

c.  Process  data: 

Fluid 

Upstream  press 
Downstream  press 
AP 

Tenperature 
Flow 

Viscosity 
Specific  gravity 

d.  Signal  data: 

Instmnent  output  signal 
Converter  -  input  signal 
output  signal 
location 

power  required 
Positioner- input  signal 


Closed 
Closed 


Max. 
Max. 
Max. 
Max. 
Max. 
Max. 
Max. 
Max. 


Min. 
Min. 
Min. 
Min. 
Min. 
Min. 
Min. 
Min. 
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output  signal 
power  required 

e.  Layaway  and  startup  data: 

ProcureEoent  reference  -  see  below 

Shutdown  reference  -  see  below 

Layaway  reference  -  see  below 

Startup  reference  -  see  below 

Location  during  layaway 

Spare  parts  for  startup  list,  part  nurribers, 
quantity,  location 

Special  tools  and  location 

Startup  priority  (a  process  decision) 

f.  Spares  and  replacenents : 

This  should  include  a  complete  list  of  parts  required 
for  normal  operation  and  for  layaway  program. 

SPEdFIGATiaj  AND  PRiXUREMEM'  INSTRUCTIONS 

These  instructions  can  be  very  sicple  or  quite  detailed  depending 
on  the  information  available.    Care  should  be  taken  to  dovetail  special 
requironents  for  a  layaway  plant  with  general  specifications  for  plants 
that  remain  in  service. 

General  Instructions  on  the  Following : 

Selection  of  body  material 

Elastomers,  i.e. ,  composition  for  actuator  diaphragms 
Plastics 

Packing,  i.e. ,  inhibitor  with  grafoil 

Actuators 

Standardization 
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Specific  Data  for  Each  Process  Solution,  Temperature,  and  Pressure 
Bocfy  and  bonnet  material 
Trim  material 
Stan  material 
Packing 
Gaskets 
Soft  seats 
Seals 

Combinations  to  avoid  galvanic  corrosion,  e.g.,  440C  and  nickel  are 
incompatible 

Plug  and  guide  materials 

Sten  and  guide  bushing  materials 

Seal  ring  and  guide 

Cobalt  alloys  are  high  resistant  to  galvanic  corrosion 
Other 

SHUTDOWN  PROCEDURES 

Prepare  standard  shutdown  procedures  covering  the  following: 

Flushing  instructions 

Draining  and  drying  instructions 

Storage  instructions 

lAYA^Y  PROCEDURES 

Prepare  a  series  of  standard  layaway  procedures  for  each  different 
set  of  conditions,  covering  the  following: 

Is  the  valve  to  be  ronoved  or  left  in  the  line? 

To  what  extent  will  the  valve  be  taken  apart? 
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Cleaning  instructions.  Solutions  required,  areas  to  watch,  e.g., 
resnove  pockets  of  liquid  in  stuffing  box. 

Protective  coatings  canpatability  with  process  fluids ,  removal 
instructions,  etc. 

Parts  to  be  stocked  for  reassembly,  e.g.,  gaskets,  packing,  soft 
seats,  actuator  diaphra^ns. 

Instructions  for  maintaining  stock  of  parts  with  limited  shelf 
life. 

Reassembly  instructions  (if  stored  unassecdbled) . 

Type  of  location  during  layaway.    Special  instructions,  e.g., 
teiperature,  moisture,  vibration. 

Personnel  involved,  training,  magnitude  of  task. 

Type  of  tools  required.    Data  on  special  tools. 

STARTUP  PRiXEDURES 

Prepare  a  series  of  standard  starti^  instructions.  The  following 
items  should  be  covered: 

Priority 

Elapsed  time  for  preparation 
Rehabilitation  data 

Tools  required 

Personnel  required 

Size  of  task 

Renoval  of  protective  coatings,  etc. 
List  of  replacement  parts 

Assembly  instructions,  i.e.,  bench  initial,  hydrostatic 
test,  seat  leakage,  hysteresis 

Installation  instructions 

Tools  required 

Personnel  required 
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Procedures,  precautions 


Data  on  line  leakage  test 
System  test  procedure 

This  sumnarizes  my  inmediate  thoughts  on  the  matter  of  layaway 
programs  for  control  valves .    I  trust  this  outline  will  help  those  v^o 
have  no  formal  programs  and  stimulate  additional  thinking  on  existing 
programs . 
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TEffi  INDEPENDENT  TESTING  AND  PRCfVING  OF  HARDWARE 


AND  SOFTWARE  ELEMENTS  OF  PLANT  CCMPLTTER  CONTROL 
SYSTEMS  ON  REACTIVATIC^  OF  MUNTTiaJS  PLANTS 


TNTRODUCTM  , 

There  has  traditionally  been  a  separation  between  hardware  and 
software  in  coirputer  control  systons.    Typically,  different  people — often  ir 
entirely  separate  organizations — are  responsible  for  the  hardware  than 
for  the  software.    These  two  groups  usually,  by  default,  test  each  other's 
work.    What  is  needed,  however,  is  independent  testing  of  both  the 
hardware  and  the  software  to  insure  proper  operation  under  all  possible 
conditions.    The  overall  system  must  be  stiructured  in  a  manner  to  make 
this  possible,  and  in  an  optimal  way. 

I  reconnend  integrating  a  judicious  philosophy  of  independent  test- 
ing with  a  five-part  approach  to  optimize  the  reactivation  of  canputer- 
controlled  munitions  plants.  j 

Modularize  the  systen. 

Design  for  start-up. 

Use  applicable  standards. 

Design  in  reliability. 

Separate  design  and  inplementation. 


SYSTEM  MJDULARITY 

Modularity  is  very  inportant  in  any  sizeable  system.    The  overall 
task  must  be  broken  down  into  manageable  sections.    Both  hardware  and 
soft^Tare  need  to  be  modularized.    Each  module  can  then  be  tested 
independently  before  it  is  integrated  into  the  total  systan.    It  should 
be  possible  to  perform  the  same  or  similar  tests  -  again  independently  - 
for  continued  maintenance  after  the  system  has  been  installed. 

Properly  structured  modularity  will  permit  optimizing  the  hardware- 
software  tradeoffs  in  system  design  (1) .    That  is ,  the  total  system 
should  be  structured  and  modularized,  then  the  decisions  made  as  to 
\^ch  sections  are  to  be  inplenented  in  hardware  and  vMch  are  to  be  in 
software. 

I  find  it  inconceivable  that  any  major  production  facility  will  be 
designed  and  built  once  and  not  have  changes  made  to  it.    As  new 


64 


technology  makes  better  performance  possible,  it  is  essential  that 
appropriate  inprovements  be  made  to  the  original  design.    Again,  a 
properly  modularized  system  will  allow  such  changes  to  be  made  with 
minimum  cost  and  minimum  effect  to  the  remainder  of  the  plant.  This 
includes  future  substitution  of  software  for  hardware,  or  vice  versa. 

There  are  many  ways  to  slice  a  pie.    Likewise,  modularity  may  be 
obtained  in  a  number  of  different  ways.    Functional  modularity  offers 
many  significant  advantages  over  other  methods  (2) ,  particularly  in  re- 
gard to  testing  and  maintenance  of  the  system.    Take,  for  example,  the 
instrumentation  for  a  tenperature  signal,  such  as  from  a  thermocouple 
(t/c) .    Such  instrumentation  could  be  modularized  in  the  following 
manner: 

'  Cold  junction  reference 

Multiplexing  with  other  analog  signals 

Analog-to-digital  conversion 

Digital  multiplexing  into  the  ccoiputer 

j  Signal  linearization  of  the  t/c  signal  (software) 

However,  this  method  generates  a  variety  of  relatively  arbitrary 

j  intermodule  signals  of  viMdi  the  specification  and  testing  can  quickly 

I  get  out  of  hand.    A  better  approach  is  to  use  total- function  instrumen- 

I  tation  \diere  a  single  instrument  module  processes  the  thermocouple  input 

i  into  a  digital  toiperature  reading  for  the  coipjter.    Now,  any  problan 

I  with  readings  frctn  a  given  t/c  can  be  readily  handled  with  single  parts 

I  substitution  (3). 

1  Software  can  also  be  modularized  in  many  ways .    The  classical 

I  method  of  structuring  process  inputs  and  outputs  is  as  follows  (4) : 

j  Analog  Inputs 

I  Analog  Outputs 

Digital  Inputs 

Ij  Digital  Outputs 

However,  these  are  hardware-related  divisiais,  ^en  we  really  need 
I     software  divisions  (5,6).    The  software  portion  of  the  system  does  not 
I     care  -  nor  should  it  need  to  know  -  how  the  hardware  is  inplonented. 
i    The  modularity  of  the  system  should  always  provide  adequate  information 
j  hiding. 

I  For  example,  a  liquid  flew  transducer  may  transmit  a  proportional 

analog  d.c.  signal  (e.g. ,  4  to  20  mA) ,  a  pulse-width  modulated  contact 

li 
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closijre  (e.g.,  3  to  12  seconds  per  15- second  cycle),  a  variable  frequen- 
cy piilse  txain,  a  parallel  BCD  encoded  data  ward,  or  a  string  of  ASCII 
characters.    Analog  input?    Digital  input?    To  the  software,  it  is  j\jst 
"input  a  proportional  variable". 

A  better  division  for  software  modularity  is  by  software  function. 
For  exanple: 

Input  an  integer  value 

Output  an  integer  value 

Input  a  logical  value  (single  bit  on/off) 

Output  a  logical  value 

This  structure  allows  the  software  to  be  tested  independent  of  the 
hardware  implementations. 


DESIGN  FOR  START-UP 

The  designers  of  manufacturing  facilities  often  think  of  a  one-time 
start-up  and  give  the  majority  of  their  attention  to  the  continuous 
operation  of  the  plant. 

When  he  vTas  young,  J.  C.  Penny  was  taught  the  i 
most  inportant  shirts  to  sell  are  the  last 
two  in  the  box  -  that's  ichere  the  profit  is. 
However,  if  you  don't  sell  the  first  ten 
shirts,  you  won't  have  any  "last  two". 

Similarly,  the  profit  in  a  ccninercial 
production  facility  occurs  during  its 
operation.    But  if  you  can't  get  the  plant 
started,  you  won't  have  any  production 
at  all. 

In  time  of  national  crisis ,  the  time  T^en  a  munitions  plant  on 
layaway  suddenly  becomes  needed,  the  highest  priority  is  to  get  started. 
(After  it  is  going  you  can  turn  your  attention  to  keeping  it  going) . 
Design  first  for  start-up. 

Even  plants  that  are  already  in  operation  have  occasion  to  stop, 
vdiether  by  desire  or  otherwise.    The  time  to  restart  a  plant  inpacts  the 
overall  reliability  and  usefulness  of  the  facility. 

The  telephone  industry,  with  its  modem  carputer- controlled  elec- 
tronic switching  systans  (ESS),  has  a  design  objective  of  no  more  than 
two  (2)  hours  total  ciranulative  downtime  in  the  40-year  life  of  an 
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installation.    The  early  ESS  offices  experienced  downtime  rates  of  12 
hours  per  projected  40-year  life  -  not  good  enough.    Bell  Laboratories 
found  they  could  reduce  the  downtime  by  50%  -  just  by  making  software 
changes. 

No,  software  does  not  fail,  at  least  it  does  not  wear  out  or  break 
down  as  hardware  equipment  does.  Once  software  is  working  correctly  it 
keeps  on  doing  that.  However,  software  (and  hardware)  can  fail  to  meet 
design  objectives,  as  in  the  case  of  the  telephone  ESS  offices.  By  re- 
designing the  software  start-up  sequences  significant  reduction  in  the 
mean  time  to  repair  (METR)  was  achieved  for  the  telephone  operation, 
entirely  independent  of  the  system  hardware. 

One  ccmputer  coipany  sold  a  large  system  to  a  customer  ^o  had 
specified  unique  keying  for  each  different  type  of  circuit  card  in  the 
systsn.    When  it  came  time  for  acceptance  tests,  the  customer  walked  in, 
renoved  every  card  from  the  system  and  stacked  them  in  a  huge  pile  in 
the  middle  of  the  rc^m.    Then  they  proceeded  to  pick  \jp  cards  at  random 
and  try  until  they  found  a  slot  in  which  the  card  would  fit.    After  all 
cards  had  been  inserted  -  somewhere  -  they  asked  the  builder  to  turn  the 
power  on.    (The  manufacturer  asked  for  a  two-day  delay  vjhlle  they  check- 
ed the  card  locations  for  proper  ke3ring).    Design  for  start-up,  not  just 
continued  operation. 

Test  for  start-up.  Test  software  initilization  sequences.  Test 
hardware  warm-iq)  times .  Test  the  start-Lq)  time  after  making  repairs , 
i.e.,  after  installing  spare  circuit  cards  or  spare  sensors. 


USE  APPLICABLE  STANDARDS 

A  key  to  sxjccessful  modularity  is  the  use  of  standard  definitions 
for  the  interfaces ,  or  boundaries ,  between  modules  (7) .  An  industry 
standard  diat  is  applicable  in  the  hardware  area  is  CAMAC*(8 , 9 , 10) . 
CAMAC  is  a  modular  instrumentation  system  intended  for  centralized 
j    digital  control  of  plug- in  instrumentation  units.    A  number  of  plug- in 
j    units  share  cannon  pcwer  supplies  and  cOTitroller  (t5T3ically  a  micro- 
processor or  ccoputer)  in  a  cccmon  assoribly  (crate).    The  CAMAC  standards 
include  both  packaging  and  digital  interface  specifications  sufficient 
to  insijre  modules  will  fit  together  regardless  of  manufacture. 

I  The  basic  CAMAC  standard  (IEEE  583)  is  further  augnented  by  IEEE 

I    595  (serial  highway)  and  tfrf.  596  (parallel  highway)  which  readily  allow 
the  centralized  control  to  be  extended  to  handle  multiple  crate 
assaiiblies. 

CAMAC  is  in  widespread  use  throughout  the  world  in  nuclear  research, 
astronony,  and  other  laboratory  applications,  as  well  as  in  industrial 
production  control  systans.    NASA  has  studied  the  use  of  CAMAC  for  the 
Space  Shuttle  program.    As  a  result,  the  CAMAC  standards  are  currently 

*  CAMAC  -  Computer  Automated  Measurement  and  Control 


being  used  to  develop  equipment  for  pay  load  instrumentation. 

In  the  software  area  a  note^le  standard  that  may  be  applied  is 
the  programning  language  FORTRA^f*'^(ll) .    In  each  case  the  standards 
provide  valuable  design  interfaces  between  system  modules .    This  permits 
different  project  teams  to  proceed  independently  of  each  other,  each 
working  towards  conmon  reference  points. 

In  addition  to  providing  program-to-program  independence  (Fortran) 
and  instrument- to- instrument  independence  (CAMAC) ,  the  combination  of 
the  two  provide  even  more  powerful  independence,  as  was  illustrated  at 
Purdue  IMiversity  a  few  years  ago. 

The  International  Purdue  Workshop  on  Industrial  Conputer  Systems 
had  already  been  working  on  standards  for  process  control  conputers  for 
a  nurrber  of  years  T^en  it  was  decided  the  attendees  would  benefit  from  a 
live  dauonstration  of  the  actual  Workshop  results.    One  month  prior  to 
the  Workshop  meeting,  t±ie  idea  was  conceived  to  dononstrate  software 
transportability,  using  ANSI  X3. 9-1966  Standard  Fortran,  xd.th  ISA-S61.1 
external  procedures  for  process  control  input  and  output,  and  to  point 
out  the  need  for  and  potential  of  standards  in  other  areas.    Two  com- 
plete ccoputer  systems,  including  peripherals  and  process  control  instru- 
mentation equipment,  were  borrowed  frcm  a  dozen  sources  throughout  the 
United  States.    The  last  major  cocponent  was  uncrated  at  Purdue  on  a 
Monday  afternoon.    Two  coiplete  systens,  with  interchangeable  process 
I/O  equipment,  and  using  the  same  source  program  deck,  were  running 
Wednesday  morning  (1,5,6). 

All  process- signal  instrumentation  used  the  CAM^  digital  interface 
standard.    This  enabled  the  same  process  instrumentation  hardware  to  be 
used  with  each  conputer  system,  as  well  as  to  make  programs  using  Fortra 
and  the  ISA-S61.1  procedures  truly  transportable. 

The  ISA  S61.1  procedures  had  been  developed  around 
the  Fortran  standards  so  as  to  be  caipiter  indepen- 
dent.   However,  because  of  the  I/O  hardware-dependence 
of  the  S61.1  procedures,  it  wasn't  until  they  were 
ccnibined  with  the  computer- independent  hard&rare 
standards  (CA^^)  that  the  applications  software 
actually  became  cccputer  independent. 

The  CAMAC  system,  by  the  way,  provides  a  standardized  "Read  Module 
Identification"  function.    CAMAC  modules  may  be  inserted  into  any  posi- 
tion in  a  crate,  the  only  difference  between  positions  is  the  address. 
The  module  identification  feature  permits  the  start-up  software  to 
check  for  proper  module/ location  match,  similar  to  t±ie  keying  of  circuit 
cards  in  older  equipment. 

One  inportant  consequence  of  using  standards  must  not  be  overlooke( 
Every  module,  every  design  team,  must  toe  the  mark.    A  weakness  in  one 
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module  camot  be  conpensated  in  the  module  on  the  other  side  of  the 
standard  interface.    This  makes  independent  testing  not  only  possible, 
but  essential. 

This  is  the  reason  a  set  of  Tests  of  Standard  Fortran  were  develop- 
ed and  published  by  Alcoa  (12),    They  are  distributed  by  Dr.  T.  J. 
Williams  of  Purdue  University  and  have  been  used  by  nearly  100  organi- 
zations as  independent  tests  of  Fortran  coopilers.    In  just  a  few  years 
after  the  introduction  of  these  tests,  there  was  a  noticeable  improve- 
ment in  Fortran  coipilers  available  on  the  market.    A  similar  set  of 
independent  tests  for  the  ISA61.1  procedures  is  also  available  through 
PurAie  University  (13) . 

Indep^dent  module  testers  have  been  developed  to  test  CAMAC 
modules  for  adherence  to  the  standard  interface  specifications  (14) . 
Independent  test  units  are  also  available  for  standard  process  signals, 
e.g.,  from  thermocoi:q)les ,  and  such  testers  should  be  used. 


DESIGN  FOR  RELIABILITY 

During  a  recent  visit  to  the  Norden  Division  of  United  Technology 
Corporation,  I  observed  a  large  sign  that  read,  "You  cannot  inspect 
reliability  into  a  product,  it  has  to  be  built  in".    I  would  add,  you 
can't  build  reliability  unless  its  in  the  design. 

Circuits  should  be  conservatively  loaded.    There  should  be  no  race 
conditions  in  the  circuit  timing.    All  software  sequences  should  be 
deterministic.    Independent  testing  is  essential. 

A  cannon  method  for  increasing  reliability  is  the  use  of  backup 
systems .    How  do  you  insure  the  backup  system  will  work  properly  when 
I     you  need  it.    (When  was  the  last  time  you  checked  the  spare  tire  in 
your  car?). .  An  operating  system  has  many  inherent  monitors.    A  standby 
systan  needs  coiparable  monitoring.    Even  on-line  redundancy  has  limita- 
tions for  achieving  reliability.    If  di^licate  circuits  produce  differ- 
ent results,  '^ich  results  do  you.  use?    Two-out- of- three  redundancy 
requires  a  voting  circuit.    What  checks  it? 

I  Anotiier  form  of  redundancy,  and  of  independent  testing,  is  the  pro- 

vision of  software  checks  for  validity  of  the  data  acquired  from  process 
sensors.    Similarly,  hardware  should  include  rate- limiting  circuits  for 
software  generated  signals. 

There  is  a  classical  conpiter  problon  in  \<hich  an 
input  of  value  one  is  to  be  changed  to  a  value  of 
two,  and  vice  versa.    But  v^t  if  the  input  is 
neither  one  nor  two  as  it  should  be?  Independent 
testing  is  needed  (15). 

I 
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Reliability  can  be  inprofved  by  including  automatic  conpensating 
circuits,  or  programs,  such  as  autanatic  zeroing  to  track  load  cell 
drift.    Independent  testing  must  be  provided  to  test  for  out-of -range 
conditions  in  the  coipensation  circuit  itself,  i.e,  vhere  it  could  no 
longer  correct  for  the  signal  drift  (16). 


SEPARATE  DESIGN  FRCM  XMPLEMENTATION 

Independent  testing  is  the  backbone  of  Quality  Assurance  programs, 
departments  are  properly  separated  from  production  departments.  It 
also  helps  to  separate  design  and  inpleanentation  teams.    That  is,  the 
implementation  group  should  be  largely  made  up  of  people  not  involved 
with  the  original  design.    This  provides  a  good  check  and  balance  -  and 
early  enough  to  make  any  necessary  adjustments  to  the  design. 

BcM  many  times  has  a  project  engineer  wanted  to  personally  check 
every  aspect  of  the  syston.    This  is  good  to  have  his  interest  but, 
assuming  he  was  involved  in  the  original  design,  it  would  be  better  to 
have  scmeone  unfamiliar  with  the  circuit,  or  program,  do  the  detailed 
implementation  and  debug. 

At  the  same  time,  continuity  is  of  great  inportance.    All  too  often 
initial  project  planners  move  on  to  newer  and  bigger  projects,  leaving 
someone  else  holding  the  bag  of  poor  specifications  that  are  impossible, 
or  at  least  impractical,  to  inplenent.    As  designers  of  closed- loop 
systems  we  must  not  forget  to  close  the  loop  on  ourselves.  Everyone 
needs  feedback  in  order  to  correct  and  inprove  future  output. 

CONCLUSION 

Independent  testing  is  a  primary  ingredient  for  achieving  success- 
ful start-Lp  or  reactivation  of  any  major  system,    ^fodularity  in  system 
structure  is  inportant  in  making  this  possible.    The  use  of  standards 
makes  all  this  practical.    Testing  cannot  create  reliability  but  it  can 
help  expose  that  reliability. 
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THE  LAXAWAY  AND  EEACTIVATION  PPDBLEMS  OF 


BEET  SUGAR  MELLS  FOR  THEIR  ANNUAL  CAMPAIQ^S 
INaUDING  MANAGEMENT,  MANPCWER  AND  TRAINING  ASPECTS 

INTRODUCTim 

A  large  part:  (257o)  of  the  sugar  used  in  t±ie  Lhited  States  cones  fron 
sugar  beets  raised  principally  in  four  regions  of  this  country,  the  mid 
west,  principally  iiorthem  Ohio;  the  eastern  Colorado-western  Nebraska 
area;  Idaho;  and  central  California.    The  wide-spread  distribution  of  thi 
growing  area  has  required  the  sugar  manufacturing  ccnpanies  to  develop  a 
large  nuriber  of  small  but  conplete  sugar  manufacturing  plants  for  this 
purpose.    Since  sugar  beets  have  a  relatively  short  storage  life  these 
plants  must  ccxiplete  the  processing  of  their  supply  of  beets  within  a 
three  to  four  month  "campaign"  starting  about  October  1  each  year  when 
the  beets  first  mature  and  are  harvested  by  the  farmers. 

Thus  the  sugar  beet  mills  must  go  through  a  deactivation,  a  layaway 
period,  a  reactivation  and  an  operational  period  each  year.    In  that 
respect  they  resecdble  the  munitions  plants  being  discussed  in  this 
Production  Readiness  Workshop.    A  description  of  their  methods  of  handl- 
ing the  several  phases  of  their  operation  and  the  reasons  for  these  is 
therefore  considered  valuable  for  the  munitions  plant  discussion. 

Figure  1  is  a  sketch  of  the  flow  paths  through  a  typical  beet  sugar  1 
mill  indicating  the  equipment  involved.    A  short  description  of  the 
processes  involved  is  given  below  (3) .    Ttiey  can  be  follov^d  on  the  dia- 
gram of  Figure  1. 

The  mills  begin  operation  in  early  October,  the  farmer  harvesting 
his  crop  with  machines  that  ijqjroot  the  beets .    Field  workers  follow  the  ' 
machines,  remove  the  dirt  and  leaves  from  the  beets,  and  load  them  for 
shipment  to  the  factory.    The  beets,  containing  frctn  13  to  17  per  cent 
sucrose  and  0.8  per  cent  ash,  enter  the  factory  by  way  of  flumes,  small 
canals  filled  with  warm  water,  vMch  not  only  transport  them  but  wash 
them  as  well. 

The  beets  are  rewashed,  weighed,  and  sliced  into  long  narrow  strips 
called  cossettes . 

The  cossettes  are  dropped  into  large  tanks  or  cells,  12  or  14  of 
which  are  connected  in  series  to  form  a  diffusion  battery.    The  sugar  is 
extracted  counter- currently  with  water  at  160  to  175  °F. ,  the  water  being 
in  contact  with  each  cell  for  6  to  8  minutes.    The  resulting  sirup  con- 
tains 10  to  12  percent  sucrose,  a  small  amount  of  invert  sugar,  and  2  to 
3  per  cent  ash.    The  pulp  remaining  contains  0.1  to  0.3  per  cent  sugar 
Cbased  on  the  beets) .    This  pulp  is  dried  and  sold  as  cattle  feed. 
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I         The  sirup  is  given  a  roijgh  screening  to  remove  foreign  materials . 

Milk  of  lime  is  added  until  the  concentration  is  equivalent  to  about 
2  to  3  per  cent.    The  lime  aids  in  the  precipitation  of  undesirable  im- 
purities.   Any  calcim  saccharate  formed  is  deconposed  in  carbonators  by 
passing  carbon  dioxide  through  thje  sirup  for  10  to  15  minutes.    The  foam- 
ing that  occurs  at  this  stage  is  redijced  by  adding  a  small  quantity  of 
soybean  oil. 

3  ij         The  sludge  produced  by  the  lime  is  equal  to  4  or  5  per  cent  of  the 
j  WBight  of  the  beets  charged.     This  is  removed  by  filtering  the  mixture 
on  Oliver  filters. 

1  I         Lime  is  added  again  mtil  the  concentration  is  equivalent  to  0.5  per 
j  cent  and  the  mixtijre  again  carbonated,  this  time  hot. 

I  It  is  then  filtered  on  a  plate-and- frame  press. 

'  i|         The  resulting  filtrate  contains  a  large  concentration  of  calcium 

ions,  v\Mch  are  removed  by  bubbling  sulfur  dioxide  through  the  sirup  in 
J  j  sulfitors.    At  the  sarne  time  the  sulfur  dioxide  serves  to  bleach  the 
j  solution  of  its  pale  yellow  color. 

•  ^         The  precipitate  of  calcium  sulfite  is  removed  on  plate-and- frame 

,  presses. 

1 

j         The  purified  sugar  is  concentrated  from  10  to  12  per  cent  sugar  to 
:    about  55  per  cent  sugar  in  raultiple-ef f ect  evaporators .    This  increases 
the  concentration  of  calcium  ions  again  and  necessitates  retreatment  with 
sulfur  dioxide  and  another  filtration. 

The  resulting  sirup  is  grained  in  vacuum  pans,  centrifuged,  washed, 
dried  in  a  granulator,  screeded  and  packed  for  cOTCiercial  shipment. 

j  The  sirup  frcm  the  first  vacuum  pan  is  given  further  treatment  to 

i  recover  more  sugar  crystals,  but  they  are  not  pure  enough  for  market  and 
jj  are  sent  back  to  the  process  for  further  purification. 

The  sirup  remaining  after  the  several  crystallizations  is  called 
1  molasses  and  may  be  sold  to  the  various  industries .    Processes  for  the 
I  '  recovery  of  the  remaining  sugar  have  been  worked  out  and  are  used  cctnner- 
cially.    Initial  recovery  is  obtained  by  the  Steffen  process. 

1|l  In  the  Steffen  process  the  molasses  from  the  centrifugals  is  cooled 

!  I  to  10°C. ,  and  finely  ground  quicklime  added.    A  precipitate  of  tricalcium 

-  saccharate  is  formed  and  filtered  off.    This  "saccharate  cake"  is  washed 

I  and  returned  to  the  process,  where  it  is  mixed  with  the  juice  from  the 

I '  diffusion  battery,  thereby  clarifying  it  in  the  same  manner  as  the  lime 

'  vMch  this  tricalcium  saccharate  partly  replaced.    The  molasses  remaining, 

I  so-called  discard  molasses,  contains  about  60  to  63  per  cent  sugar ,  2  to 
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5  per  cent  raffinose,  and  14  to  16  per  cent  ash,  based  on  the  dry  mate- 
rial.   It  is  sold  as  livestock  feed  or  processed  for  the  recovery  of 
monosodiim  glutamate  and  other  amino  coipounds. 

OPERATION  OF  ^  SUGAR  BEET  MILL 

As  mentioned  above  the  sugar  beet  mill  operates  on  a  3-4  month 
"canpaign"  each  year  to  extract  sucrose  from  sugar  beets  and  convert  it 
into  coomercial  and  table  sugar.    The  remaining  8-9  months  of  the  year, 
the  "intercarapaign"  in  sugar  conpany  parlance,  the  plant  is  idle.  It 
must  therefore,  each  year,  go  through  a  deactivation  or  shutdown,  a  laya- 
way,  and  a  reactivation  or  start-up,  just  as  the  munitions  plant  being 
considered  in  this  Workshop.    The  major  difference,  of  course,  is  that  it 
is  known  ahead  of  time  that  the  sugar  beet  mill  must  be  restarted  each 
fall.    Ihus  planning  covers  this  occurrence.    In  addition,  the  relatively 
short  "layaway"  or  inactive  period  is  used  for  any  maintenance  or  re- 
modelling work  needed  on  the  mill  prior  to  its  being  restarted.  No 
planning  need  be  done  for  an  indefinitely  long,  totally  inactive  period 
vMch  might  occur  vdth  the  munitions  plant. 

Table  I  outlines  the  deactivation  or  shut  down  procedures  followed 
in  the  beet  sugar  mill.    Table  II  outlines  work  during  the  inactive 
period  and  Table  III  covers  restart  or  reactivation  operations.  The 
degree  of  repair  work  mdertaken  and  the  amount  of  renovation  planned 
during  the  inactive  period  is ,  of  course ,  a  matter  of  available  ccnpany 
budgets. 


PERSONNEL  POLICIES 

During  the  campaign  a  typical  beet  sugar  mill  operates  on  a  staff  of 
about  150  persons.    All  management  and  supervisory  personnel,  skilled 
tradesmen  such  as  maintenance  men,  and  year-round  tasks,  such  as  watch- 
men, constitute  a  permanent  staff  which  are  full-time  enployees.  All 
others  are  part-time  ecployees  hired  only  for  the  campaign  period.  The 
permanent  staff  amounts  to  about  45-56  persons  distributed  as  sho^n  in 
Table  IV.    The  permanent  staff  carries  out  as  much  of  the  plant  repair, 
renovation,  and  product  warehousing  work  as  possible  during  the  inter- 
campaign.    Additional  personnel  are  hired  to  carry  out  as  much  of  this 
work  as  necessary  to  assure  its  ccopletion  before  the  next  campaign. 

Maintaining  all  supervisory,  maintenance,  and  key  operational  person- 
nel on  permanent  enployment  does  cut  down  considerably  on  any  training 
programs  necessary  for  part-time  enployees  hired  only  for  the  canpaign 
period.    The  major  training  problem  facing  plant  management  is  the  train- 
ing of  laboratory  assistants  needed  to  test  intermediate  and  final  prod- 
\acts,  to  monitor  plant  operation  and  assure  product  quality.  Regular 
training  sessions  must  be  held  for  this  purpose  since  they  are  mainly 
local  house-wives  hired  only  for  this  purpose  and  turn-over  is  high. 
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TABLE  I 


ACTIONS  (M  DEACTIVATION  OF  A  BEET 
SUGAR  MILL  AT  THE  END  OF  A  CAMPAIGN 


1.  All  process  imits:  tanks,  lines,  punps,  etc.  are  disassecdbled  and 
cleaned  vath  water.    Those  units  \^ch  have  accurmilated  lime  scale 
are  cleaned  first  with  acid  and  then  witii  water. 

2.  All  punp  icnpellers  are  checked  for  wear,  any  \\Mch  are  excessively 
vrom  are  marked  for  repair  during  the  inactive  period.    All  bearings 
are  checked  for  wear  and  repaired  and  repacked  if  necessary. 

3.  All  exposed  or  smooth  surfaces  of  punps,  valves,  etc.  are  coated  with 
grease  or  oil  to  prevent  rust. 

4.  All  transducers,  valves,  etc.  are  disassembled,  checked  and  repaired 
as  necessary.    Parts  are  stored  in  a  dry  place  to  prevent  any  corro- 
sion during  the  inactive  period. 

5.  After  cleaning,  all  instrument  boards,  power  panels,  etc.  are  covered 
with  polyethylene  sheets  to  keep  out  dirt  and  excess  moisture. 

6.  Tanks  are  coated  with  sugar  water  or  soybean  oil  to  reduce  any 
incipient  corrosion. 


78 


TABIE  II 

OPERATIONS  DURING  THE  INACTIVE  PERIOD 
FOR  EEET  SUGAR  MILLS 

1.  Repairs  are  made  on  all  imits  vMch  require  replaconent  or  repair 
prior  to  the  next  canpaign. 

2.  Units  requiring  repainting  are  taken  care  of  during  this  period. 

3.  Any  alternations  planned  for  the  plant  are  acccnplished. 

4.  In  high  humidity  areas,  those  pieces  of  equipment  subject  to  damage 
frcm  corrosion  are  stored  in  air  conditioned  areas . 

5.  Bearings  of  all  large  equipment  are  turned  weekly  to  avoid  flat- 
spotting. 
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TABLE  III 


OPERATIONS  DURING  THE  REACTIVATION  PERIOD 
FOR  BEEI  SUGAR  mUS 


1.  Iftiits  are  cleaned  of  protective  coatings  applied  diBring  the  de- 
activation period. 

2.  All  disassenfcled  units,  transducer,  valves,  punps,  motors,  etc.  are 
reassenbled.    All  valve  stems,  inpellers,  bearings,  etc.  are  repacked 
as  necessary.    Packings  are  saturated  v/ith  oil  and  graphite. 

3.  Once  the  plant  has  been  reassecdbled,  it  is  operated  with  plain  water 
to  assure  that  all  joints  are  tight,  that  all  lines  are  clear,  and 
that  all  instruments,  pumps  and  other  components  are  operating 
correctly. 
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TABLE  IV 


PERSONNEL  COUNTS  FOR  A  TYPICAL  BEET 
SUGAR  MILL  DURING  THE  INTERCAMPAIGN 
(I.E.,  PERMANENT  STAFF) 

1.  Management  Personnel 

1  Factory  Manager 

1  ^fester  Mechanic 

1  Factory  Manager's  Clerk-Stenographer 
3 

2.  Jfeintenance  Staff 

4  Ass't.  >fester  >fechanics 

8  Mechanics 

8  Repairmen 
20 

3.  Operational  Personnel 

3  Si:pervisors 

9  Foremen 

8  Operators 

1  Clerk 
21 

4.  Watchaen,  etc. 

2  Watchmen 

5.  Total  -  46 
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Likewise  operators  assistants  must  be  briefed  by  the  operators  during  the 
reactivation  period  and  the  water  test  phase  of  initial  operations. 


IMPLIGAIIONS  FOR  MUI^^:TIONS  PLAOTS 

The  scheduled  reactivation  of  the  beet  sugar  mill  does  appreciably 
change  the  deactivation  and  inactive  period  operations  in  conparison  to 
the  munLtions  plant.    However,  it  is  felt  that  the  experiences  of  the 
beet  sugar  plant  are  valuable,  particularly  their  steps  to  inspect,  repair 
and  preserve  all  plant  components  at  the  end  of  a  canpaign.    Thus  the 
munitions  plant  should  be  treated  in  a  similar  manner  -  the  only  differ- 
ence being  more  thorough  preservative  measures  because  of  the  indefinite 
length  of  the  inactive  period.    It  is  felt  that  all  items  should  be  in  a 
perfect  state  of  repair  before  storage  to  assure  the  best  possible  state 
in  the  beginning  of  reactivation. 
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INTRDDUCriON 


The  reduction  in  prices  of  minicon^niter  systems  and  the  availability 
of  a  variety  of  microprocessors  and  related  digital  hardware  have  made 
distributed  digital  systens  both  feasible  and  affordable  in  industrial 
applications.    These  systans  have  a  number  of  failure  modes  and  perfor- 
mance characteristics  v^ch  are  different  frcm  previous  generations  of 
control  hardware,  and  therefore  require  new  reliability  design  techni- 
ques.   However,  these  programnable  systems  also  have  capabilities  for 
self-diagnosis  and  fault  tolerance  x^ch  could  not  even  be  considered  in 
earlier  hardwired  systems.    In  this  paper,  the  characteristics  of  dis- 
tributed digital  systems  T^tiich  are  relevant  to  system  reliability  are 
described.    Also,  a  number  of  reliability  design  techniques  which  can  be 
used  in  the  development  and  implementation  of  the  syston  architectures 
are  described.    While  this  paper  was  prepared  specifically  to  discuss 
the  problans  vMch  exist  in  the  deactivation  and  activation  of  munitions 
plants,  the  techniques  surveyed  also  can  be  used  in  other  applications 
of  distributed  digital  systems. 


A  DISTRIBUTED  SYSTEM  FOR  MUNTTIONS  MANUFACTURING 

One  of  the  ways  in  vMch  munitions  manufacturing  facilities  can  be 
improved  is  through  the  use  of  the  "latest  proven  manufacturing  tech- 
nology", as  described  in  Attachment  I  to  the  Workshop  directives.  The 
latest  technology  to  appear  on  the  manufacturing  scene  is  digital  medium- 
scale  and  large-scale  integrated  circuitry,  v^ch  has  been  used  in  pro- 
grammable controllers ,  conpjter  information  and  monitoring  systans ,  and 
digital  controllers  for  individual  units  of  production  machinery.  Cur- 
rently, a  number  of  electronic  equipment  manufacturers  are  using  this 
technology  to  develop  distributed  digital  systems  for  control  and  moni- 
toring of  chemical  processes,  electrical  power  generation  and  distribu- 
tion systans,  and  discrete  manufacturing  processes. 

One  possible  configuration  of  such  a  distributed  system  in  a  muni- 
tions manufacturing  plant  is  illxistrated  in  Figure  1.    In  this  plant, 
there  are  a  nmiber  of  relatively  independent  production  lines,  each  of 
vdiich  receives  parts  or  subasseniblies  from  (or  provides  than  to)  other 
lines.    In  general,  there  is  a  sufficient  amount  of  storage  space  for 
inventory  between  the  lines  so  that  the  shutdown  of  one  line  for  a  short 
time  would  not  shut  the  whole  plant  down.    In  this  case,  it  makes  sense 
from  a  system  reliability  point  of  view  to  provide  a  separate  page  of 
hardware  to  control  each  production  line  individually.    In  Figure  1, 
this  hardware  is  called  a  "Line  Control  Unit"  or  LCU.    Each  LCU  can  per- 
form any  or  all  of  the  following  functions : 

1)  Closed- loop  modulating  control  (e.g.,  control  of  continuous 
melt-pour  of  INT) 

2)  Closed- loop  sequence  control  (e.g. ,  sequential  machine  tool 
control) 
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3)  Monitoring  of  data  for  record-keeping  or  safety  assurance 
purposes . 

4)  Provision  of  a  local  interface  to  an  operator  for  manual 
control  or  supervision  of  the  production  line. 

5)  Transmission  of  data  to  a  central  control  room. 

6)  Acceptance  of  control  ccninands  (such  as  set  point  inputs, 
production  rate  comnands,  etc.)  from  a  central  plant  operator 
interface  or  computer  syston. 

In  addition,  each  LCU  has  the  following  features: 

1)  Any  failure  in  an  LCU  does  not  affect  the  operation  of  any 
other  LCU. 

2)  Each  LCU  is  responsible  for  only  one  production  line  or  unit 
(or  a  portion  of  a  production  line,  if  the  line's  conplexLty 
is  hi^) . 

3)  Each  LCU  has  its  own  "intelligence"  in  the  form  of  one  or  more 
minicccQputers  or  microprocessors,  in  addition  to  conventional 
hardwired  equipment  (e.g.,  sensors,  actuators,  controllers, 
hardwired  logic). 

As  indicated  in  Figure  1,  a  Plant  Operator  Interface  Syston  may  be 
provided  to  allow  an  operator  in  a  central  control  room  area  to  monitor 
the  performance  of  the  total  plant  and  to  take  control  actions  as  appro- 
priate.   This  system  may  include  both  conventional  panelboard  instrumen- 
tation as  well  as  CRT  (cathode  ray  tube)  displays ,  and  would  be  driven 
by  its  own  coapiter  system  or  processors.    Also,  a  Plant  Conpiter  System 
may  be  supplied  to  execute  various  plant  performance  calculations,  diag- 
nostic routines,  or  production  optimization  functions.    The  various 
elanents  of  this  distributed  system  then  are  connected  by  a  digital 
Plant  Ccnmunication  System,  \^ch  allows  control  ccnmands  and  other  data 
to  be  transmitted  among  the  individual  "intelligent"  systan  elements. 

The  operation  of  munitions  plants  \^ich  are  subject  to  layaway  can 
be  divided  into  three  phases:    dormant  phase,  startup  phase,  and  opera- 
tional phase.    In  the  dormant  phase,  the  system  has  been  deactivated  and 
is  in  layaway.    Periodic  tests  of  system  operational  readiness  may  be 
required  to  identify  dormancy  failures  and  initiate  module  repairs  or 
replacements.    The  startup  phase  occurs  after  mobilization  and  may  in- 
volve a  sequenced  initialization  of  the  individual  LCUs,  the  computer 
system  and  display  system.    Detailed  diagnostic  tests  at  all  levels  of 
the  hierarchy  may  be  required  to  verify  the  operational  readiness  of 
each  system  element  before  the  next  step  in  the  initiation  sequence  is 
taken.    Finally,  ^en  the  operational  phase  is  reached,  the  munitions 
plant  is  run  in  the  same  way  as  are  other  continuous  manufacturing 
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processes.    The  types  of  techniques  used  to  ensure  the  reliability  and 
safety  of  this  systan  will  be  different  at  each  phase  of  the  plant '  s 
operation.    In  the  next  Section  some  of  these  techniques  are  discussed,    ]  ifei 
in  the  context  of  the  distributed  systan  architecture  shown  in  Figure  1. 
These  techniques  are  described  in  more  detail  in  the  following  Section. 


RELIABILITY  FEATURES  OF  DISTRIBUIED  DIGITAL  SYSTEMS 

The  structure  and  performance  characteristics  of  distributed  digi- 
tal systans  present  both  problems  and  opportunities  to  the  manufacturing 
systan  designer.    The  failure  modes  and  rates  of  mechanical,  pneumatic, 
and  discrete  electronic  devices  have  been  established  through  long  years 
of  e2q)erience;  therefore,  they  can  be  taken  into  account  in  the  design 
process  to  provide  high-reliability  systans  with  acceptable  failure 
modes.    As  described  by  Boyd  (44)  and  Bryant  (32),  the  state  of  reliabil- 
ity design  knowledge  in  the  case  of  processor-based  systems  is  not  nearly 
as  well  advanced.    Thus,  the  designer  must  be  more  careful  to  follow 
proven  reliability  design  principles  and  be  perhaps  more  conservative  in 
estimating  the  failure  rates  of  systan  elanents. 

The  two  main  advantages  that  distributed  digital  systans  provide 
the  designer  in  the  area  of  reliability  are:    1)  the  ability  to  detect 
actual  or  icpending  failures  more  easily  through  die  use  of  diagnostic 
programs,  and  2)  tJcie  ability  to  minimize  the  effects  of  failures  on  the 
total  systan  by  partitioning  the  distributed  systan  properly.  Neither 
of  these  features  would  be  possible  without  the  existence  of  inexpensive 
cocoputing  power  provided  by  the  minicoiputer  and  microprocessor.  Diag- 
nostics and  self- checking  programs  can  be  added  to  these  devices  with 
almost  negligible  increase  in  cost.    Also,  the  low  cost  of  the  processors 
allows  each  to  be  assigned  only  a  small  portion  of  the  total  functions 
of  a  systan,  thus  minimizing  the  impact  of  a  single  processor  failure. 
In  a  centralized  or  integrated  control  systan  (such  as  a  direct  digital 
control  systan  in  a  central  conputer),  there  is  a  danger  that  the  total 
systan  will  fail  if  a  single  elanent  in  the  systan  fails .    In  a  distri- 
buted systan  architecture  such  as  that  shown  in  Figure  1,  a  failure  in  a 
single  LCU  affects  only  that  LCU  if  the  systan  design  has  been  done 
properly.    (See  Reference  5  for  a  ccoparison  of  centralized  and  distri- 
buted systan  reliability).    Similarly,  a  failure  in  the  central  cooputer 
or  operator  interface  systan  (vdiich  are  at  the  top  of  the  systan  hier- 
archy) does  not  affect  the  operation  of  the  LCUs  (which  are  at  the  bottom 
of  the  hierarchy).    To  ensure  that  this  is  the  case,  a  detailed  Failure 
Modes  and  Effects  Analysis  (FMEA)  must  be  performed  during  the  system 
design  phase  to  determine  the  effect  of  all  likely  failures  of  individual 
systan  elements .    Another  useful  design  technique  ^^ch  minimizes  the 
likelihood  of  catastrophic  failures  is  the  Cocmon  Mode  Failure  Analysis 
(Q^) ,  in  \h±ch  systan  elanents  having  similar  mechanical,  electrical  or 
other  properties  are  grouped  together  and  analyzed  for  cannon  failure 
sources.    A  related  technique  which  is  used  to  identify  potential  criti- 
cal failure  modes  in  a  hazardous  environment  (certainly  characteristic 
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of  a  munitions  plant)  is  the  Fault  Tree  Analysis  (FIA.)  approach.  In 
this  method,  all  possible  undesired  events  (such  as  explosions)  are 
identified.    Then,  the  combinations  of  "gated"  events  vibxch  could  lead 
to  the  undesired  event  are  identified  and  analyzed  for  probability  of 
occurrence.    (Note:    The  underlined  techniques  mentioned  above  are 
described  in  more  detail  in  the  next  section) . 

When  the  miinitions  plant  is  in  its  dormant  phase,  periodic  tests  of 
system  elements  may  be  useful  in  minimizing  plant  starti:q5  time  by  identi- 
fjring  failures  before  mobilization  occurs .    A  technique  known  as  Test 
Interval  Analysis  (TIA)  may  be  useful  in  identif5dng  the  maxiimin  time 
interval  between  \^ch  tests  should  be  made  to  guarantee  a  given  level 
of  system  reliability.    This  approach  allows  the  designer  to  perform  an 
economic  tradeoff  between  the  frequency  of  tests  and  the  resultant  re- 
liability of  the  syston. 

The  partitioning  of  the  distributed  system  shown  in  Figure  1  also 
I  allows  the  selective  introduction  of  redundant  hardware  at  the  siibsystem 
level  if  reqioired  to  perform  critical  functions.    This  results  in  a  more 
j  cost-effective  system  with  equivalent  or  better  reliability  than  a  system 
I  having  redundancy  at  die  total  system  level.    Redundancy  can  be  intro- 
I  duced  at  the  following  levels: 

1)     hardware  within  each  Line  Control  Unit 

I  2)     total  Line  Control  Unit  redundancy 

I  3)     Plant  Comnunication  System 

1  4)     Plant  Operator  Interface  System 

j  5)     Plant  Computer  Systan 

I  Of  course,  one  of  the  key  issues  in  Redundancy  Design  is  the  approach 
J  iised  to  ccmbine  or  switch  the  outputs  of  several  redundant  channels  to 
'   perform  a  single  function.    Some  of  these  approaches  are  described  in 
the  next  section. 

,  The  reliability  design  techniques  mentioned  above  are  standard  ones 

1  vAiich  have  been  used  in  the  past  in  hard-wired  system  design  as  well  as 
I  being  appropriate  to  stored-program  digital  systems.    Other  techniques 

have  been  developed  more  recently  and  are  more  suitable  to  the  latter 
.1   class  of  systans .    Fault  Tolerant  Design  Techniques  are  used  to  ensure 
j   that  a  systan  can  continue  to  operate  aespite  mcmentary  or  permanent 
|(   equipment  failures.    In  the  distributed  system  shown  in  Figure  1,  these 
j   techriiques  would  be  applied  to  the  design  of  the  Line  Control  Units  and 
I   Plant  Ccnmunication  System.    As  indicated  previously,  the  availability 
|i    of  stored  program  logic  allows  the  use  of  more  sophisticated  Failure 

Detection  Techniques  or  fault  diagnostics  than  were  possible  previously. 
I   These  techniques  can  be  applied  to  the  design  of  both  the  Line  Control 
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Units  at  the  low  end  of  the  hierarchy  as  well  as  to  the  Plant  Computer 
Systan  at  the  high  end.    The  many  advantages  of  stored  program  devices 
are  partially  offset  by  the  new  problsn  of  ensuring  that  the  software 
viiich  defines  the  operation  of  these  devices  is  accurate.    While  soft- 
ware design  can  be  controlled,  reviewed,  and  documented  in  the  same  way 
as  is  hardware,  its  relatively  nebulous  nature  increases  the  tenptation 
to  short-cut  sound  design  procedures.    As  a  result,  a  number  of  Software 
Reliability  Design  Techniqiies  have  been  developed  to  inprove  the  quality 
of  software  being  generated  for  distributed  digital  systems . 


RELIABILITY  DESIGN  TECHNICS 

The  techniques  mentioned  briefly  above  are  described  in  more  detail 
in  this  section.    More  information  on  these  techniques  is  contained  in 
the  references  list  in  the  Bibliography  at  the  end  of  the  paper. 

REDUNDANCY  TECHNIC^S 

Redundancy  is  a  system  design  method  used  to  inprove  reliability  by 
providing  back-up  equipoient  to  replace  the  function  of  failed  equipment. 
Redundancy  offers  the  greatest  inproveoent  in  reliability  when  the 
system  can  be  repaired  without  shutdown.    Care  must  be  taken  viien  speci- 
fying a  type  or  level  of  redundancy  because  in  sane  cases  it  can  actually 
degrade  system  reliability.    Redundancy  is  also  a  costly  design  method 
because  it  requires  a  duplication  of  hardware,  and,  in  some  instances, 
additional  software.    The  cost  effective  design  uses  no  more  redundancy 
than  required  to  achieve  the  required  reliability  objectives.    There  are 
numerous  redundancy  techniques.    The  type  employed  depends  on  the  spe- 
cific application.    The  following  are  some  of  the  more  generally  used 
methods: 

1)  Simple  Parallel  Redundancy  -  The  duplication  of  functionally 
similar  cccponents /modules  (channels)  with  no  switching  or 
isolation  between  channels.    This  method  is  Qi5)loyed  when 
the  failure  mode  of  a  failed  channel  is  such  that  it  does 
not  alter  the  operation  of  the  remaining  good  channels. 
This  method  could  be  used  to  provide  multiple  paths  between 
LCUs  in  the  Plant  Comnunication  System  shown  in  Figure  1. 

2)  Simple  Series  -  Parallel  Redundancy  -  The  duplication  of 
functionally  similar  co^)onents /modules  with  a  series 
isolation  element.    This  method  is  employed  v^enever  the 
failure  mcxie  of  a  failed  channel  is  such  that  it  does 
alter  the  operation  of  the  remaining  good  channels,  but 
the  failed  channels  can  be  detected  and  isolated  by  a 
siii5)le  series  eloiient.    This  method  is  used  in  power 
systODs  in  \^ch  power  supplies  are  provided  in  parallel 
and  isolation  is  obtained  through  an  auctioneer  panel. 
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3)  Stand-by  Parallel  Redundancy  -  The  duplication  of  functionally 
similar  conponents /modules  with  one  or  more  channels  in  an 
off-line  mode.    This  method  requires  a  series  switching 
element.    It  is  enployed  vtien  the  failure  mode  of  a  channel 
could  alter  the  operation  of  the  remaining  good  channels 

and  vAien  the  failure  of  a  channel  can  be  tolerated  for  the 
period  of  time  required  to  switch  out  the  failed  channel 
and  switch  in  the  stand-by  cliamel.    This  method  is  used  in 
computer  systans  in  vMch  (for  example)  multiple  peripherals 
of  the  same  type  are  provided,  and  the  switching  device  is 
the  control  section  of  the  computer. 

4)  Voting  Redundancy  -  The  duplication  of  functionally  similar 
ccnponent s  /modules  with  a  coipare  circuit  vMch  monitors 
the  output  of  all  channels  and  which  accepts  the  majority 
as  the  correct  output.    This  method  requires  at  least  three 
channels.    It  is  aiployed  v^en  the  failure  mode  of  a  channel 
could  be  any  one  of  several  states  and  the  failed  states  are 
possible  normal  states.    This  method  is  used  in  the  safety 
systans  for  nuclear  power  plants  and  in  a  nucdber  of  safety- 
related  military  applications. 

5)  Analytical  Redundancy  -  The  dLplication  of  functionally 
similar  caiponents /modules  with  a  ccnpare  circuit  which 
monitors  the  output  of  all  channels,  and  determines  when  a 
failure  has  occurred.    It  also  requires  an  algorithm  \diich 
determines  the  failed  channel  and  bypasses  it.    This  method 
is  enployed  v^en  the  failure  mode  of  a  channel  coiild  be  any 
one  of  many  states  and  the  failed  states  are  possible 
normal  states.    It  could  be  used  to  provide  redundancy  in 
the  sensors  used  in  the  munitions  plant  distributed  control 
system.    This  method  typically  requires  less  hardware  than 
the  voting  method;  the  difficulty  is  obtaining  an  algorithn 
that  can  detect  and  isolate  a  faulty  diannel. 

A  related  method  which  is  useful  in  the  study  of  redundancy  is  the 
Reliability  Block  Diagram  (RED) .    This  is  an  analysis  technique  which 
presents  pictorially  the  success  path  of  the  system.    Its  purpose  is  to 
identify  all  possible  success  paths  leading  to  satisfactory  operation  of 
the  system.    The  RED  does  not  consider  human  interactions  with  the 
system.    Rather,  it  is  used  to  identify  areas  within  the  system  that  re- 
quire redundancy. 

The  RED  is  constructed  as  follows: 

1.  Define  systan  failure  and  siiccess. 

2.  Prepare  a  list  of  all  systen  conponents/modules . 

3.  Prepare  a  list  of  all  possible  success  paths. 
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4.  List  for  each  success  path  all  of  the  components /modules 
required  for  the  path. 

5.  Present  pictorially  in  logical  form  all  of  the  success 
paths.    Use  synibols  to  represent  each  ccnponent /module. 


FAILURE  M3DES  AND  EFFECTS  ANALYSIS  (FWEA) 

The  IWEA  is  an  analysis  technique  used  to  identify  credible  modesl 
of  failijre  and  to  evaluate  their  consequence.    Any  system  employing  re- 
dundancy or  fault  location  techniques  must  have  knowledge  of  the  possible 
failure  modes  of  each  coiponent /module  and  their  effect  on  the  system. 
Redundancy  would  be  detrimental  if  the  only  possible  channel  failure 
mode  conpromises  all  channels.    Fault  location  is  possible  only  ^en  one 
knows  all  of  the  failure  effects  and  vMch  conponents/modules  could  cause 
them.    The  FMEA  method  outlined  below  was  extracted  from  IEEE-SrD-352- 
1975. 

The  PMEA  is  performed  as  follows: 

1.  Prepare  a  list  of  all  System  Components/Modules. 

2.  For  each  Component /Module,  list  the  following: 

a)    Possible  credible  failure  mode;  for  each 
failure  mode,  list  the  following: 

i.    Possible  failure  mechanisms 

ii.    Symptoms  and  local  effects  (including 
dependent  failures) 

iii.  Method  of  detection 

iv.  Inherent  compensating  provision 

V.  Effect  on  syston 

vi.  Remarks  and  other  effects 


Cmm  M3DE  FAILURE  ANALYSIS  (CMA) 

The  0^  is  an  analysis  technique  used  to  detect  multiple  failures 
attributable  to  a  conmon  cause.    Systems  stored  for  long  periods  in  the 
dormant  state  may  contain  some  devices  approaching  or  exceeding  their 
design  life.    Some  examples  are:    shelf  life  of  aluminum  capacitors,  de- 
gradation of  printed  circuit  boards  due  to  humidity  or  dirt  and  grease 
accumulations,  and  corrosion  of  open  contacts.    In  addition,  operating 
transients  and  maintenance  anrors  m^  result  in  multiple  feiluEEScf  similar 
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conponents.    In  systans  employing  redundancy,  multiple  similar  failures 
can  result  in  ijnsafe  systan  operation;  especially  in  those  instances 
\Aiere  the  failed  state  is  a  possible  normal  state.    The  CMA.  method  out- 
liaed  below  was  extracted  from  IEEE-STD-352-1975. 

The  CMA.  is  performed  as  follows : 

1.  Prepare  a  list  of  all  System  Ccnponents /Modules .  (List  A) 

2.  Prepare  a  list  of  units  each  of  \^ch  is  composed  of  all 
conponents/modules  identical  or  similar  in  design  or 
construction. 

3.  Prepare  a  list  of  units  conposed  of  all  components /modules 
that  have  identical  function  and  are  not  conpletely  diverse. 

4.  Perform  IMEA.  on  units  identified  above. 

5.  Examine,  as  a  minimum,  the  following  considerations  for  all 
failure  modes  T^ch  cause  primary  system  failijre  or  fails 
secondary  systans: 

a)  Random  failure  (List  A  only) 

b)  Determine  the  credible  sources  of  failure  due  to: 
manufacturing  process 

environmental  factors 
maintenance  or  installation  errors 
operator  interactions 

operating  transients  and  credible  accidents 

FAULT  TREE  ANALYSIS  (FTA) 

The  FIA  is  an  analysis  technique  that  organizes  deductively  and 
presents  pictorially  all  actions  that  can  contribute  to  an  undesired 
event.    Its  purpose  is  to  identify  all  credible  sources  of  an  undesired 
event.    The  technique  is  an  extension  of  both  the  FMEA  and  the  Reliabil- 
ity Block  Diagram  (RED)  in  that  it  considers  all  actions,  not  just 
equipment  malfunctions.    It  is  different  frcoi  the  RED  in  that  it  is  a 
failure  diagram  instead  of  a  success  diagram.    Startup  and  operation  of 
most  systems  require  some  man-machine  interactions.    Undesired  events, 
such  as  explosions,  are  to  be  avoided  at  all  costs.    The  FIA  allcws  one 
to  ascertain  all  of  the  possible  human  and  machine  actions  that  can  lead 
to  the  ev^t.    The  FTA  pinpoints  areas  requiring  redundancy.    The  FTA 
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method  outlined  below  was  extracted  from  TTW.-STD-352-1975. 

The  Flk  is  performed  as  follows  for  each  mdesired  event: 

1.  Define  undesired  event. 

2.  Define  the  main  branches  that  can  lead  to  the  undesired  event. 

3.  For  each  main  branch  deduce  the  chain  of  subevents  back  to 
the  fundamental  event  (source) . 

4.  Document  the  analysis  via  logic  syiribols. 

TEST  PTTERVAL  ANALYSIS  (TIA) 

The  TIA  is  a  method  used  to  determine  the  test  interval  necessary 
to  meet  a  specified  system  reliability.    Any  system  operating  in  a  dor- 
mant state  and/or  en^jloying  redundancy  requires  testing  to  achieve  a 
high  level  of  reliability.    Equipment  standing-by  in  a  donnant  state 
still  possesses  a  finite  failure  rate.    Even  ^en  redundancy  is  aiployed, 
testing  is  still  required  to  achieve  a  high  level  of  availability. 
Periodic  testing  can  impact  the  systan  to  the  extent  that  sane  redund- 
ancy may  not  be  required  or  a  lower  level  may  be  sufficient  to  meet  a 
specified  reliability.    In  most  large  systems  there  will  be  an  infinite 
cccibination  of  test  intervals  on  the  various  system  elonents  which  will 
satisfy  the  reliability  requironent.    An  analysis  is  required  to  deter- 
mine a  satisfactory  cccnbination. 

The  TIA  is  performed  as  follows: 

1)  Prepare  a  list  of  all  System  Conponents/^dules 

2)  Determine  an  estimate  of  the  failure  rate  of  each 
ccmponent/module  for  both  the  operational  and  dormant 
modes. 

3)  Perform  an  REA 

4)  Either  construct  an  RED,  or: 

5)  Construct  an  equation  which  mathematically  represents  the 
reliability  of  the  systen.    The  equation  is  a  function  of 
failure  rate,  repair  rate,  test  interval  and  operate  time. 
It  is  a  mathanatical  representation  of  the  RED.    It  is 
developed  using  probability  techniques  such  as  Bayesian, 
Markov,  or  Ifonte  Carlo. 

6)  Solve  the  equation,  at  the  specified  reliability  level,  for 
the  test  interval.    Engineering  judgement  and  a  judicious 
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choice  of  initial  parameters  will  result  in  a  satisfactory 
solution. 


FAILLfRE  DETECnON  TECHNIQUES 

There  is  a  need  to  detect  faults  or  failures  in  a  distributed 
digital  system  for  a  nuniber  of  reasons: 

to  shut  down  a  portion  of  the  process  so  that  the  field 
control  equipment  will  not  cause  an  unsafe  condition 

to  inhibit  startup  of  a  portion  of  the  process  until 
the  failed  control  equipment  has  been  repaired  or 
replaced 

to  initiate  a  transfer  to  a  redundant  piece  of 
control  equipment 

to  revert  to  a  manual  mode  of  process  operation  vMch 
bypasses  the  failed  control  equipment. 

Equipnent  in  a  digital  system  generally  falls  into  three  catagories: 
processors,  manories,  and  other  hardwired  equipment  modules.    The  fail- 
ure detection  approaches  used  depend  on  the  category  considered.  Some 
processor  failiire  detection  techniques  are: 

Test  programs  -  Programs  with  known  inputs  and  outputs 
are  run  by  the  processor  on  a  regular  basis;  if  the  results 
do  not  agree,  the  processor  declares  itself  "sick"  and  shuts 
itself  down.    Obviously,  this  technique  works  if  there  is  a 
failure  in  only  a  portion  of  the  processor's  circuitry.  This 
technique  also  provides  a  partial  check  on  the  operation  of 
the  memory. 

"Watchdog"  timer  -  A  hardware  timer  external  to  the  processor 
must  be  reset  periodically  by  the  processor  at  regular 
intervals  in  its  execution  cycle.    If  it  does  not,  the  timer 
times  out  and  transmits  a  signal  announcing  that  the 
processor  has  failed.    This  approach  does  not  depend  on  the 
processor  to  check  itself. 

Techniques  for  detecting  failures  in  other  equipment  rely  on  the 
proper  operation  of  a  processor  (either  the  central  ccoputer  or  a  pro- 
cessor in  the  LCU) .    In  addition  to  the  test  program  diagnostic  described 
above,  other  memory  failure  detection  techniques  are: 

1)     Parity  checks  -  a  single  bit  is  appended  to  every  data  word 
in  memory  to  allow  checking  for  an  odd  number  of  errors  in 
storage  or  fetching  operations.    If  an  odd  parity  check  is 


1) 
2) 
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ijsed,  the  parity  bit  is  set  in  such  a  way  that  the  total 
nucdber  of  ones  in  the  data  word  plus  parity  is  odd  (in  an 
even  parity  check,  the  sum  is  even).    Then,  \dien  the  processor 
fetches  a  word  frcm  memory,  it  conpares  the  parity  bit  with 
the  total  nuiiber  of  ones  in  the  word  to  see  if  an  error  exists. 
If  it  does,  the  processor  flags  that  portion  of  memory  as 
having  failed  (perhaps  after  one  or  more  retries  to  fetch 
the  word) . 

2)     Sum  checks  -  The  words  within  a  specified  block  of  memory  are 
added  together  at  systen  generation  time  and  the  sun  is 
stored  at  the  end  of  that  block.    Periodically,  the  processor 
reconputes  the  total  and  conpares  it  to  the  sum  previously 
stored  to  identify  any  failures  in  that  memory  block.  This 
technique  is  useful  only  for  read-only  manories  (ROMs)  or 
for  random  access  memory  (RAM)  containing  a  fixed  set  of 
data  or  instructions.    It  is  an  effective  technique  for 
dedicated- function  microprocessor  systems  using  ROMs. 

Failure  detection  techniques  for  other  equipment  modules  fall  into 
two  categories: 

1)  Processor-directed  diagnostics  -  These  are  programs  run  by 
the  processor  to  detect  module  failxjres  by  exercising  the 
modules  and  comparing  the  actual  results  with  expected 
results.    For  example,  one  way  to  test  input/output  modules 
in  an  LCU  is  to  provide  a  "wrap-around"  hard-wired  connection 
frcm  an  output  module  to  an  input  module.    The  processor  then 
writes  a  known  signal  value  to  the  output  module,  reads  the 
corresponding  chamel  on  the  input  module,  and  conpares  the 
two.    If  they  do  not  agree  (to  within  the  accuracy  specifi- 
cations of  the  modules) ,  there  is  a  failure  in  the  I/O  module 
string. 

2)  Module- level  diagnostics  -  Each  module  can  provide  a  binary 
output  signal  vhich  indicates  its  state  of  health  and  \diich 
is  made  available  to  the  processor  for  use  in  shutdown 
initiation,  switchover  to  a  redundant  module,  or  other 
purposes.    The  status  signal  is  derived  from  internal  checks 
on  lODdule  power  status ,  operational  activity  (the  watchdog 
timer  approach  also  can  be  used  here) ,  and  other  indicators 
of  module  "health".    The  specific  checks  made  will  depend 
on  the  type  of  module  (e.g.,  input,  output,  controller, 
ccranunications  interface)  and  its  level  of  "intelligence" 
(seme  modules  may  contain  their  own  dedicated  processors) . 

The  various  failure  detection  techniques  described  above  can  be 
implemented  in  the  plant  central  computer  as  well  as  in  the  LCUs  during 
all  phases  of  plant  operation.    Since  the  most  likely  maintenance  ap- 
proach in  this  system  is  simple  failed  module  replaconent,  the  detection 
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techniques  used  should  not  attenpt  to  isolate  failures  below  the  module 
level. 


FAULT  TOLERANT  DESIQ^  TECHNIQUES 

Fault- tolerant  design  features  allow  a  systen  to  continue  operation 
despite  momentary  or  permanent  hardware  failures,  electrical  noise 
transients,  or  other  system  faults.    The  performance  of  the  systan  (e.g., 
I    time  responsiveness,  data  throughput  capability)  may  be  degraded  to  some 
extent,  but  the  basic  functions  to  be  performed  are  still  accoctplished. 
Some  of  the  fault- tolerant  design  techniques  used  in  distributed  digital 
systems  are  as  follows: 

1)     Fault- tolerant  system  partitioning  -  To  the  maximiin 

extent  possible,  the  individual  modules  and  subassemblies 
in  the  system  are  designed  in  such  a  way  that  their  failure 
does  not  affect  the  rest  of  the  system.    This  fault  indepen- 
dence is  then  verified  by  means  of  a  failure  modes  and 
effects  analysis.    The  chief  limitation  of  this  approach 
is  usually  the  cost  involved  in  providing  separate  services 
(such  as  electrical  power,  cooling,  etc.)  to  the  individual 
elements  in  the  systen. 

"Load- sharing"  systan  design  -  In  this  approach,  the  systan 
functions  are  assigned  to  the  available  facilities  in  such 
a  way  that  the  performance  load  is  shared  equally  by  the 
facilities,  with  no  facility  being  dedicated  to  a  particular 
function.    This  is  t±ie  approach  used  in  banks  which  have 
multiple  tellers  to  service  a  single  queue.    The  loss  of 
any  one  teller  results  in  a  degradation  in  queue  servicing 
rate,  but  not  in  a  loss  of  any  banking  function.    The  digital 
system  equivalent  is  a  multiprocessor  system  in  vMch  several 
processors  share  a  coranon  memory  and  perform  tasks  assigned 
to  them  in  real  time  rather  than  at  system  generation  time. 
If  properly  designed,  these  systems  provide  "graceful  degrada- 
tion" in  case  of  failure,  since  the  loss  of  a  single  processor 
affects  throughput  but  is  not  catastrophic.    However,  these 
systems  often  require  operating  system  software  which  is 
quite  cai5)lex  and  vdiich  contains  a  large  amount  of  overhead 
in  assigning  tasks,  managing  memory,  and  performing  fault 
diagnostics.    This  type  of  systan  is  also  quite  susceptible 
to  single-point  hard^rare  failures  or  software  errors  which 
could  cause  the  entire  system  to  fail. 

3)     Fail-safe  output  hardware  -  When  hazardous  processes  (such  as 
munitions  plants)  are  being  controlled  by  automatic  digital 
or  analog  equipment,  great  care  must  be  taken  to  ensure  that 
an  equipment  failure  does  not  cause  an  unsafe  process  condi- 
tion.   For  this  reason,  outputs  from  the  control  equipment  are 
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usually  interfaced  to  the  process  through  fail-safe  hardware. 
This  hardware  is  designed  so  that  a  failure  in  it  or  in 
i:pstreain  equipiient  causes  the  control  outputs  to  revert  to 
eitter  a)  a  "safe"  state,  if  such  a  state  can  be  identified 
or  b)  the  last  valid  output  state.    In  the  case  of  processor- 
based  control  systems,  two  approaches  are  canronly  used: 

a)  Outputs  from  the  processor  are  contact  outputs  which 
increnent  or  decrement  an  external  memory;  a  processor 
failxire  causes  the  outputs  to  remain  in  their  previous 
state.    They  can  then  be  manipulated  through  manual 
inputs  from  an  operator  station. 

b)  The  output  hardware  requires  that  the  processor  actively 
switch  a  contact  or  reset  a  watchdog  timer  before  the 
processor's  control  outputs  are  transmitted.  Otherwise, 
a  standard  "safe"  set  of  outputs  are  used. 

Additional  discussions  of  these  techniques  are  given  in 
References  32  and  37. 

4)     Use  of  parity  and  error  correction  codes  -  In  a  distributed 
digital  system,  data  transfers  occur  across  internal  parallel 
buses  (such  as  between  a  processor  and  its  memory)  as  well  as 
across  serial  conmunication  links  (such  as  between  LCUs  over 
a  Plant  Connunication  System) .    Various  codes  are  used  to 
check  die  accuracy  of  these  data  transfers  and  either  a) 
indicate  that  an  error  has  been  made,  in  vMch  case  the 
transfer  must  be  repeated;  or  b)  use  the  capabilities  of  the 
code  to  correct  the  error,    thus  making  the  transfer  tolerant 
of  faults  in  the  transmission  mechanism.    The  former  case  was 
covered  in  the  discussion  on  error  detection  techniques  in 
Section  -"Failxjre  Detection  Techniques".  In  the  latter  case  of 
error  correction,  hardware  in  the  data  transmitter  appends  an 
additional  check  field  of  bits  to  the  basic  data  word  or 
message.    The  check  field  contains  the  resiolts  of  several 
different  parity  calculations  based  on  various  bit  ccoibinaticre 
in  the  data  field.    When  the  expanded  data  message  is  received, 
the  receiving  processor  uses  the  contents  of  the  check  field 
plus  an  additional  set  of  parity  cccqputations  to  determine 
the  location  (not  just  the  existence)  of  errors  in  the  data 
field.    It  can  then  correct  these  errors  by  reversing  the  bit 
status  at  these  locations  and  continue  operation  without 
repeating  the  data  transfer.    In  essence,  the  check  field 
provides  redundant  information  in  the  data  transfer  to  allow 
reconstruction  of  the  data  field  mder  a  limited  set  of  error 
conditions.    This  approach  is  rarely  used  in  industrial 
systems  due  to  the  inefficiency  (and  resulting  cost)  of  in- 
cluding the  redundant  information  in  each  data  transfer. 
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As  mentioned  above,  one  of  t±ie  problecos  in  designing  processor-baaed 
systans  is  that  of  guaranteeing  that  the  software  or  programs  vAiich  de- 
fine the  processor's  functions  is  correct,    Tlie  methods  vMch  have  been 
developed  to  help  ensijre  that  the  software  is  correct  are  loosely  grouped 
under  the  name  of  "software  reliability"  design  techniques.    This  phrase 
is  somendiat  of  a  misnomer,  because  software  in  itself  cannot  fail,  any 
more  than  a  wiring  diagram  or  circuit  diagram  can  fail.    Rather,  it  is 
either  correct  ^en  implenented  or  it  is  incorrect.    A  true  failure  can 
occur  only  in  the  hardware  with  vMch  the  software  is  inplauented.  If 
no  hardware  failure  occurs,  a  piece  of  software  may  only  appear  to  fail 
Vivien  the  particular  conditions  viiich  esqpose  its  incorrectness  occur, 
either  through  following  a  program  branch  containing  an  error  for  the 
first  time  or  through  encountering  a  set  of  input  conditions  v^tilch  ex- 
pose the  "failure". 

A  number  of  techniques  have  been  developed  to  improve  the  quality 
or  "reliability"  of  software,  including  the  following: 

1)  Structured  Ihrogramraing  -  The  flexibility  of  program  structure 
allowed  the  programner  is  restricted  so  that  the  resultant 
code  is  more  amenable  to  autanatic  checks  for  errors. 

2)  Use  of  special  high-level  languages  -  The  syntax  of  high-level 
languages  can  be  designed  to  reduce  the  likelihood  of  software 
errors  and  to  simpliify  their  detection  through  autanatic  means. 

3)  Off-line  software  checks  -  These  are  static  analysis  proced- 
ures  vMch  check  program  structures  for  blockages,  loops,  or 
other  execution  flow  problems,  and  make  consistency  checks  on 
the  \ise  of  machine  resources  and  peripherals. 

4)  On-line  software  checks  -  These  are  dynamic  analysis  tools 
(sometimes  called  "software  audits")  vtiich  verify  the  accuracy 
and  consistency  of  the  contents  of  volatile  memory  and  the 
integrity  of  register  linkages  associated  with  facilities, 
among  other  things.    These  tests  are  executed  periodically 

on  a  "snapshot"  of  the  system  state  at  one  time,  and  are 
designed  to  consume  only  a  small  fraction  of  the  system 
resources . 

5)  "Proofs  of  correctness"  -  The  structure  of  the  program  itself 
is  analyzed  off-line  to  determine  whether  or  not  it  meets  a 
set  of  validation  conditions.    The  analysis  procedure  is 
similar  to  that  used  in  proving  mathematical  theorems,  and  is 
amenable  to  automation.    The  validation  conditions  also  can 
be  checked  on-line  to  verify  proper  program  operation. 
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These  techniques  are  covered  in  more  detail  in  References  38  to  42 . 
They  are  not  strictly  reliability  techniques  in  the  classical  sense,  but 
they  may  be  useful  in  in5)roving  the  accuracy  of  the  software  in  distri- 
buted digital  systems. 

ADDmONAL  INFORMATION 

The  subject  of  higji-reliability  system  design  is  an  extremely 
broad  one,  as  can  be  seen  from  the  range  of  techniques  discussed  above. 
Only  the  key  concepts  \^ch  are  most  applicable  to  the  munitions  plant 
layaway  problem  have  been  described  here.    Additional  pertinent  informa- 
tion can  be  obtained  through  the  references  listed  in  the  Bibliography  as 
well  as  fran  the  literature  on  related  research  and  development  areas 
vMch  use  high-reliability  design  techniques.    Some  of  these  related 
areas  are  as  follows: 

1)  Studies  of  equipment  failures  during  dormancy  conditions 
conducted  by  the  Air  Force  at  the  Rome  Air  Development 
Center  (see  Reference  49 ,  for  example) . 

2)  Studies  of  the  storage  reliability  of  conponents  used  in 
the  U.S.  Army  missile  program  (see  Reference  47,  for 
exacnple) . 

3)  Development  of  safety  systems  for  nuclear  power  plants. 

4)  Design  of  redundant  canputer  systans  for  air  traffic  control 
and  railroad  dispatching  applications  (see  Reference  44, 
for  example) . 

5)  Use  of  fault- tolerant  digital  systems  in  other  military 
and  space  applications  (e.g.,  ICBM  program,  unnamed 
NASA  deep  space  probes,  space  shuttle,  and  fly-by-wire 
aircraft  control  systens) . 

Each  of  these  areas  has  its  own  particular  reliability  and  safety 
requirements ,  and  each  should  be  examined  to  identify  the  specific  desigi 
techniques  vMch  can  be  adapted  for  use  in  distributed  digital  systems 
for  munitions  plants. 
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USE  OF  SIMULATION  AND  OTHER  RELATED 


TECHNIQUES  FOR  THE  TRAINING  OF  STARTUP 
CREWS  FOR  REACTIVATION  OF  LAID-AWAY  PLANTS 


INIRODUCTION 

Arthur  C.  Clark  in  his  motion  picture  and  book,  "2001  :  A  Space 
Odyssey",  projected  that  a  team  of  skilled  astronauts  would  be  main- 
tained in  suspended  animation  aboard  traveling  space  vehicles  until 
the  time  arrived  for  future  operations  to  begin.    At  a  pre-deterarLned 
time/ space  threshold  they  would  be  gradually  revived  and  made  ready 
I  to  conduct  the  intricate  phases  of  the  mission.    This  is  an  interest- 
ing and  unique  solution  for  meeting  a  rather  extended  need.  While 
j  there  is  a  simularity  of  need  associated  with  the  future  operation  of 
I  earth-bound,  reactivated  production  facilities,  a  vtolly  different  set 

of  conditions  apply  and  so  some  other  kind  of  solution  must  be  provided. 
I  For  PROJECT  LAYAWAY  it  is  a  package  of  technical  know-how  that  is  to  be 
'  preserved  -  and  in  such  a  form  that  it  can  be  acquired  readily  by  a 
I  future  team  assenbled  for  operations  whenever  the  need  arises.  Later, 
'  we  may  want  to  debate  vAiether  the  challenges  of  establishing  a  life 
S'i:5)port  syston  or  of  preparing  a  training  mechanism  is  the  easier  to 
satisfy. 

A  startup  crew  has  to  be  oriented  to  the  system  it  is  expected  to 
operate  and  given  a  means  of  developing  the  particular  skills  vMch  will 
I  assiire  successful  operation  of  the  plant.    A  proper  approach  involves 
I  appreciating  the  characteristics,  features,  and  environment  of  the  pro- 
jj  cfuction  system,  the  attributes  of  the  human  operator,  and  the  techniques 
i  available  to  us  now  and  in  the  near  future.    An  integrated  design  \^ch 

I  combines  automation  technology,  simulation  techniques,  and  hunan  factors 
!  considerations  can  permit  an  incorporation  of  the  necessary  intangible 

j  elements  coopatibly  with  the  tangible  parts  of  the  production  system, 
jl  I  propose  to  review  methods  and  components  \^Mch  can  be  exploited  to 

II  serve  these  purposes. 

j  SYSTEM  FEATURES  AND  ENVIRONMENT 

'  I  submit  that  the  most  capable  type  of  system  design  for  this 

i  project  is  one  that  enploys  an  electronic,  modularized,  conputer-based, 

j  consolidated  instrumentation  console  for  operations  control.  This 

I  choice  (over  conventional,  panel-mounted,  parallel  instruments)  is 

i  appropriate  because  it  affords  an  extensive  medium  for  on-line  training 

I  activities  that  can  be  given  directly  in  the  context  of  the  processes 

'I  involved.    VJhile  it  may  be  premature,  I  am  assuming  for  this  discussion 

I  that  such  a  configuration  vrould  be  selected. 
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Typical  of  the  process  monitoinjng  and  control  consoles  \^^ich 
recently  have  been  brought  to  market  is  the  Taylor  WT)  III  System  de- 
picted in  Figure  1.    The  distributed  modularization  allows  opportuni- 
ties for  incorporating  additional  itens  which  can  hold  and  present 
training  information  and  responses  coordinately  with  the  primary  dynam- 
ic process  traffic.    Looking  more  closely  at  the  console  in  Figure  2, 
three  aspects  of  the  system  can  be  seen.    A  colorgraphic  display  of  a 
unit  operation,  a  continuously  updated  status  display  of  t±ie  processing 
equipment  and  conditions,  and  a  display  of  selected  variables  showing 
trends  as  they  transpire  over  the  imnediate  past.    The  fourth  bay  con- 
tains cassette  recorders  for  accimulating  dynamic  behavior  vdiich  can 
be  broiight  into  display  at  will  with  considerable  flexibility.  Anno- 
tated detail  of  these  bays  are  shown  in  Figures  3,  4,  and  5. 

A  wide  range  of  training  functions  can  be  accctnnodated  by  extending 
such  a  system  in  three  ways.    First,  a  mass  memory  added  to  the  system 
bus  can  retain  tutorial  program  material  and  lesson  responses  accessible 
through  the  third  bay  during  periods  T^hen  the  process  is  on  target  or 
is  not  in  operation.    Second,  an  audio  facility  can  be  added  for  carry- 
ing coded  tone  alerts  and  textual  cooinentary .    Third,  specially  designed 
circuit  modules  can  be  appended  to  t±ie  control  rack  so  that  upon  ad- 
dressing prograiiinable  transfer  switches,  selected  portions  of  the 
process  (including  the  total  process)  will  be  simulated  by  dedicated 
hybrid  cctiputer  channels. 

So  as  not  to  encumber  the  principal  control  consoles  unnecessarily, 
an  auxiliary  console  should  be  located  nearby  for  syston  diagnostics, 
repair,  and  calibration  tasks.    When  not  so  needed  this  separate  console, 
fitted  with  a  video  tape  recorder  (VTE.) ,  can  serve  as  the  base  for  ini- 
tial orientation  sessions  and  allow  free  access  to  preliminary  simu- 
lation exercises. 

The  learning  and  working  environment,  then,  is  established  such 
that  the  operator  is  seated  before  an  array  of  console  displays  suit- 
ably human  engineered  for  conpatible  lighting,  comfortably  reachable 
buttons ,  and  modulated  sound  for  effective  attention  (An  optional  high- 
quality  headset  should  be  available  with  local  speaker  nuting  to  enhance 
concentrated  following  of  lessons  and  process  behavior.)  Adjustable 
thresholds  assigned  to  selected  process  signals  are  invoked  automati- 
cally v^enever  training  activity  is  initiated  so  that  an  imnediate 
interruption  can  override  in  the  interest  of  critical  conditions. 


THE  TRAINING  TASK 

Over  the  years  the  training  of  operators  in  the  process  industries 
has  undergone  numerous  developments  and  trials.    Operators  receive 
general  instruction  in  scheduled  classes  at  the  plant  site  and  are  then 
exposed  to  the  particulars  of  a  given  manufacturing  conplex  on-the-job. 
Of  course,  considerable  time  is  required  to  leam  in  this  manner  and 
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the  performance  results  are  rarely  uniform  among  individuals.  Also, 
v^en  an  operator  is  transferred  to  another  assignment,  his  training 
i  does  not  remain  with  the  unit  and  some  other  person  must  undergo  skill 
development. 

In  order  to  iiiprove  this  situation  and  anticipate  functional  needs 
of  new  or  modified  processes,  electronic  conputer  models  have  been  used 
to  simulate  process  dynamics  and  characteristics  for  training  purposes. 
Usually  operators  must  be  sent  to  the  simulation  laboratory  for  exposure 
sessions  vibBre  the  association  with  the  subject  process,  unfortunately, 
is  missing.    One  of  the  best  techniques  used  for  training  involved 
connecting  an  analog  conputer  model  of  a  process  to  the  actual  instru- 
ment control  panel  at  the  plant  site  using  mutiplexed,  telephonic  tele- 
1  metry  (by  means  of  PFM,  piAsed  frequency  modulation)  .    The  operators 
became  conditioned  to  their  system  through  the  same  interface  they 
would  continue  to  serve.    While  successful,  this  method  is  cunfoerscsne 
1  and  expensive  so  it  is  not  employed  often.    A  conmercial  firm  (1)  can 
i  sijpply  the  same  technique  in  a  separate,  free-standing  cabinet  vMch 
j  can  be  transported  from  plant  to  plant.    Of  course,  it  also  incurs  quite 
ij  an  investment.    As  of  today  the  training  situation  at  most  plants  is 
I  still  in  need  of  vast  improvement. 

At  the  recent  Sumner  Conputer  Simulation  Conference, Killian  re- 
I  ported  (2)  that  current  fuel  and  operating  costs  for  ccranercial  aircraft 
j  have  led  the  U.S.  airlines  to  set  a  goal  for  training  flight  crews 

I  totally  throijigh  simulation.    As  the  trends  shown  in  Figure  6  for  his 

j  COTpany  show,  they  are  near  reaching  that  goal.    Their  facilities  are 
'i  very  extensive  and  continually  being  improved.    The  nature  and  style 
i  of  modem  cockpits  lend  themselves  to  reflected  simolation  nuch  more 
[!  conpletely  than  do  process  control  rooms.    Only  with  the  latest  console 
Ijj  designs  has  our  area  of  operation  offered  a  reasonable  chance  to  under- 
I|  take  a  broadly  applied  iraprovesnent  in  operator  training. 

rl        A  ccoputer-based  system  provides  a  medium  to  use  self- tutoring, 
! ;  progranmed  instruction  integrated  with  simulation  exercises .    Going  at 
:  his  own  pace  the  operator  is  trained  to  deal  with  each  section  of  the 

process  in  turn  as  he  works  across  a  series  of  instructional  modules. 

Each  section  begins  with  a  visual  and  aural  description  of  the  process 
i I  purpose,  design,  functional  features,  and  operational  technique  — 

I I  all  accompanied  with  automatic  placement  of  distinctive  cursors ,  color 
'  shifts,  and  response  trends.    Nbrmal  operation  is  illustrated  along 

i  with  system  tolerances  and  constraints.    Ujpsets  are  inposed  to  shew  how 
!  overrides  carry  the  process  autonatically  to  intermediate  plateaus . 
!|i  Finally  the  operator  is  trained  how  to  conpensate  for  unscheduled  faults 
|i  and  abnq)t  shifts  in  conditions . 

j        Cctnputer  assisted  instruction  (CAI)  has  been  promoted  and  used  in 
I  the  educational  comiunity  for  some  years  with  mixed  reaction  and  results . 
A  miiiber  of  factors  and  concerns  have  appeared  with  its  use.    First,  the 
i  student  finds  the  conputer  and  interface  terminal  cold  and  unrelenting  — 
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not  alv^ys  well  aligned  wit±i  the  nature  of  the  si±iject  matter.  The 
materials  require  intense  preparation  by  extremely  well  qualified 
personnel.    Simulation  aspects  must  be  inpleraented  adequately  so  as  to 
be  convincing  and  not  misleading.    The  student  must  be  motivated  and 
sustained  to  pursue  a  course  and  see  it  through  to  a  conclusion. 

With  well -prepared  materials  process  operators  are  likely  to  find 
GAI  very  acceptable  because  it  is  available  to  them  in  the  context  of 
their  job  v^ere  sufficient  incentives  can  be  maintained.    To  aid  his 
progress  the  operator  should  be  able  to  bring  into  a  view  a  functional 
sunmary  of  any  lesson  he  has  previously  covered.    Also,  each  operator 
could  be  given  a  personal  update  cassette  \(\Mch  accimulates  his  skill 
scores  and  lesson  sumnaries  (not  to  be  monitored  by  their  s\:5)ervisors 
during  the  learning  period) .    Motivation  can  be  hei^tened  by  placing 
interesting  gametronic  programs  in  memory  accessible  to  the  individual  a 
limited  number  of  times  when  certain  score  or  skill  levels  have  been 
achieved. 

To  ease  the  intensity  of  lesson  preparation  Osin  (3)  has  developed 
a  software  system  he  calls  SMXIH  (Self-directed  Mixed  Initiative 
Tutorial  Helper) .    SMXIH  relieves  the  course  author  of  the  burden  of 
computer  progranming,  thios  greatly  reducing  course  preparation  time. 
Ihe  author  writes  his  normal  instructional  text,  divides  it  into  frames 
(suitable  for  screen  presentation) ,  and  describes  each  frame  in  terms 
of  the  topics  it  covers.    He  may  also  establish  precedence  and  some 
other  relations  between  frames.    From  then  on  SMIIH  structures  the 
material  and  displays  it  to  each  student  in  a  tutorial  mixed- initiative 
mode,  tailoring  the  presentation  according  to  the  student's  performance 
and  requests.    SMITH  is,  in  fact,  an  instructional  information-retrieval 
system  enploying  special-piirpose  algorithms  in  order  to  make  intelligent 
use  of  mim'mRl  content.    In  this  way  it  is  possible  to  provide  CAI  in 
an  efficient  and  adaptive  manner  without  demanding  an  unrealistic  time 
investment  from  the  author. 


SIMUIATIQN  TECHNIQUES 

Creating  and  inplementing  the  process  models  for  training  exercises 
and  skill  development  is  the  most  challenging  part  of  this  proposed  sys- 
tem.   Before  touching  on  the  details,  let's  review  the  description  of 
the  simulation  procedure  as  published  by  Pobanz  (4) .    Simulation  consists 
of  the  several  functional  activities  illustrated  in  Figure  7.    The  first 
block  is  simply  a  definiton  of  the  physical  system  to  be  modelled.  Once 
a  system  has  been  specified,  the  next  function  is  to  analyze  the  system 
in  the  particular  areas  of  interest.    For  exairple,  in  the  case  of  a 
conpressor,  this  might  include  whether  there  is  adequate  anti- surge 
control  and  whether  there  is  any  undesirable  system  response  resulting 
from  t±ie  surge  control  method.    Witii  these  objectives  stated,  the  next 
functional  block  is  to  describe  by  a  set  of  mathematical  eqmtions  the 


110 


physical  variables  vMch  detennine  its  transient  behavior  as  related 
to  the  stated  objectives.    In  the  example  of  a  compressor,  this  would 
involve  variables  such  as  suction  pressure,  discharge  pressure,  vapor 
flow,  etc. 

Developing  good  mathematical  models  is  not  black  magic,  but  rather 
a  sound  application  of  existing  principles  and,  vhen  obtainable,  em- 
pirical data.    The  next  functional  block  involves  the  solution  of  these 
sets  of  equations.    The  proper  conputer  is  best  determined  by  each 
application.    Ihe  solution  block  produces  results  v^Mch  can  then  be 
evaluated.    The  evaluation  may  lead  to  several  courses  of  action. 
Three  of  these  shown  in  the  figure  are: 

Vary  parameters 

Re-analyze  the  math  model 

Re-design   the  physical  system 

In  the  case  of  varying  parameters  one  can  see  that  the  functional 
blocks  of  "solve"  and  "evaluate"  will  determine  how  rapidly  this  action 
can  be  accomplished.    Since  evaluation  time  is  controlled  by  the  person 

!  using  computer  simulation,  it  suffices  to  say  that  a  ccoputational 
technique  capable  of  producing  many  solutions  in  a  short  amount  of  time 
would  be  desirable.    The  hybrid  ccmputer  (coi^Dled  analog  and  digital 

1  components)  meets  this  criterion,  especially  when  connected  to  real- 

'  time  systems. 

I        The  second  course  of  action  results  if  the  evaluation  process 
)  determines  that  the  simulation  model  does  not  adequately  represent  the 
Ij  I  physical  system.    In  this  case  we  are  dealing  directly  with  the  validity 
'jof  computer  models.    The  third  course  of  action,  dictated  by  the  evalua- 
tion process,  results  in  a  conplete  re-definition  of  the  physical  system. 

To  show  hew  the  technique  is  practiced  an  actual  dual- train,  raolti- 
I'l  stage  compressor  system  is  shown  in  Figure  8.    The  describing  equations 
are  given  in  Figure  9  along  with  key  ccmponents  (indicated  in  parenthe- 
ses) fomd  in  the  prograraning  diagram  of  Figure  10. 

I        More  than  fifteen  years  ago  a  systems  engineer  said  the  following 
j  (5):    "Process  simulation  has  many  diferences  from  more  widely  publi- 
cized military  simulation  projects.    One  of  the  most  important  is  the 
fact  that  usually  little  is  known  about  the  mathematical  behavior  of  a 
particular  process.    Most  equipment  is  non- linear  as  well.    As  a  result 
one  often  finds  himself  making  an  approximate  simulation  of  an  approxi- 
mated non- linear  equation  vMch  approximately  describes  the  process". 
iAnd  just  last  year  Lee  and  Weekman  in  reviewing  control  practice  in 
the  chemical  industry  had  this  to  say  (6):    "The  single  mDst  difficult 
jliproblem  to  be  overcome  is  understanding  the  process  itself.  Chemical 
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processes  are  generally  characterized  by  large  dimensionality,  strong 
interaction  among  process  variables,  and  nonlinearity" . 

While  this  situation  is  likely  to  persist  for  propellant  manufac- 
Uiring  processes  as  well,  there  are  two  considerations  vMch  are  in 
favor  of  our  being  able  to  meet  the  training  objectives.    One  is  that 
high  accuracy  of  model  conformance  is  not  essential  to  adequately 
represent  the  process  for  training  exercises .    It  would  be  a  different 
matter  if  die  model  were  to  serve  design  efforts.    And  second,  since 
the  process  will  be  operational  prior  to  taking  it  into  the  layaway 
state,  the  opportunity  to  tune  the  models  against  actual  performance 
enpirically  will  be  afforded.    As  much  as  possible  I  would  suggest 
using  2nd-order-plus- dead  time  channels  to  match  process  dynamic  re- 
sponses and  adjustable  gain  circuits  for  coupling  parameters.  Logic 
modules  could  be  used  to  inpose  constraints  and  reflect  configuration 
changes.    The  use  of  the  control  channels  for  both  the  operations  and 
simulation  modes  will  simplify  the  modeling  and  minimize  the  added 
investment  in  instrumentation. 


CONaUSIONS 

Even  though  there  has  been  an  avalanche  of  new  electronic  products 
in  recent  years  and  outstanding  development  in  audio/visual  teaching 
methods ,  it  must  be  emphasized  that  this  reactivation  project  is  an 
ambitioijs  undertaking.    While  there  is  now  no  technical  breakthrough 
needed  to  meet  the  goals,  much  will  have  to  be  done  to  reduce  to  prac- 
tice the  potential  technology  that  is  available.    Only  by  embedding  the 
system  knowledge  within  when  tlie  total  structure  is  engineered  can  a 
successful  outcctne  be  expected. 
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MOD  III  SYSTEM 


COMMAND  CONSOLE 


CONTROL  RACK 


Figure  1. 
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MODE 
C  -  Computer 
A  -  Automatic 
M  -  Console  Manual 
L  —  Rack  Local  Manual 
Blank  —  Indication  Loops 


Last  Page 
Assigned 
Line  No. 


Controlling 


Switching 


NOTES: 

1.  Entries  appear  for  Controlling  Loops  only. 


2.  Indicates  process  point  for  Controlling  Loops  or 
Status  of  switching  loops. 


3.  Left  blank  for  Switching  Loops 


-20%  -10%  0   +10%  +20% 
Wide  Bar  Indicates 
Percent  Deviation  of 
Process  from  Set-Point 
(Note  1) 


Figure  3.    Status  Display- 


lie 


Process  Range  in 
Percent  from 
Oto  100 


iNumber  of  Hours 
overall  graph  repre- 
sents. Commonly 
called  Trend  Time  ' 
Base.  If  blank. 
Trend  Time  Base  is 
171/2  minutes. 


Time  data  was  recorded  when 
PVTR  Cassette  is  being  played  back. 
(Commonly  called  Trend  Time  Playback) 

Loop  Process  Variable  Top  of  Set-Point  Marker  /  Rack  and 
Name  Trend  Line         Range  (arrow)       /       Slot  No. 


/  / 

/ 

/ 

HR  /216          0  / 

REAC/rOR  POLYMER 


/- 

/ 

/ . 

/ 

/ 

/       IHR       217/  00- 

r 


TYPE  OF  DISPLAY 
B  -  Current  Data 

Fast  Trend 
T  -  Tape  Playback 
D  -  Slot  or  Display  Data 
K  —  Formatting 


First  digit  indicates  PVTR  No. 
and  next  two  digits  represent 
channel  No..  If  blank,  loop  is 
not  being  recorded. 


Bottom 

of 
Range 


Process  Marker 
(Bar) 


Data  being  addressed 
is  displayed  here  for 
verification  before 
being  entered. 


MODE 
Computer 
A  —  Automatic 
M  -  Console  Manual 
L  -  Rack  Local  Manual 
Blank  —  Indication  Loops 
&  ON/OFF 
Switching  Loops 

PVTR 
Recorder  and 
Channel  Number 
(Second  Loop) 


Figure  4.    Variable  Trend  Display 
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Process  PVTR 
Variable                 System  Memory  Playback 
Tape  Recorder               Tape  Recorder              System  Memory  Controls 
(PVTR)                         and                   Playback  Controls  and 
Transport              Playback  Transport           and  Indicators  Indicators 
 \  \  J   / 


\  \ 
PVTR  Tape  Drive  System  Memory 

Controls  and  Indicators  Tape  Recorder 

Controls  and  Indicator 


Figure  5.    Cassette  Recorder  Station 
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Figure  6. 
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PHYSICAL  SYSTEM 
(BASIC  OESIGNI 


ANALYZE 
ICONSTnuCT  MATH 
MODELl 


DESCRIBE 
(COMPUTER  MACHINE 
LANGUAGE) 


(COMPUTER  ACTION! 


I 


RESULT 
(DATA  OR  GRAPHIC 
PRESLNTATIONl 


EVALUATE 
(MAN  JUOGEMENTI 


(EMPLOY  RESULTS! 


Figure  7.    Slinulation  Process 
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NOMENCLATURE 


Q 

Absolute  Temperature 

Mass  Flow  Rate 

p 

Absolute  Pressure 

D 

Impel ler  Di  ameter 

K 

Specific  Heat  Ratio 

N 

Shaft  Rotation  Rate 

0 

Flow  Coefficient 

H 

Head  or  Work 

Head  Coefficient 

R 

Ideal  Gas  Constant 

Polytropic  Efficiency 

Z 

Compressibility  Factor 

M.S.D 

Subscripts  for  mean,  suction 

EQUATIONS 


d  i  scharge  conditions 


Q  = 

(Amp.    1)         Q5  = 


PQ    =      Z  m  RT 
Z  m  RT 


P 

m  Ts 


where  Cj     =  ZjR 


(Amp.   3)  0    =    ^  • 


QS  .  1 


where  C2  = 


AT    =  (Td-Ts) 
(Amp.  6)      T[)    =  Ts+<2.T 


(Pot  k) 


(Pot  5) 


AT 


N^D^ 

n2d2 


ZmR(K^'K  AT 


K-1 
KK 


KX 
K-1 


(^)         •  Pc 

Ts 


>\-  C4 


ZmR  (i^)  -  Tg)  (Amp.  9)  - 


where  Ci, 


K  

K-1 


(Pot  7) 


(Amp.  5)  AT 


N^ 


C3 


(Pot  6) 


Figure  9. 
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OPERATING  TOMY'S  PLANT  DESIGN  TOMDRRCW 


INIRQDUCrKl^ 

Plant  start-ups  are  difficult  times  even  \^ben  they  occur  iirmediately 
after  construction  has  ended  and  the  design  is  still  fresh  in  the  minds 
of  those  involved.  The  situation  is  helped  somewhat  because  at  this  stage 
the  plant  operating  objectives  can  still  be  recalled;  the  basis  for  deci- 
sions can  still  be  rerreiiibered;  and  those  directly  involved  in  the  entire 
project  are  still  available  for  consultation.    The  problem  posed  by  the 
Arny  in  its  Monitions  Production  Base  Mademization  and  Expansion  Program 
explicates  the  start-up  problem  by  introducting  a  time  delay  between  the 
initial  construction  and  the  full  plant  operation. 

j         During  the  intervening  years  vtoi  the  munition  facilities  are  in  the 
"Layaway"  state,  the  people  involved  will  certainly  change.    The  plant 

I  operators  are  not  the  only  ones  to  be  considered,  but  others  such  as  the 
designers,  managers,  supervisors,  contractors  and  vendors,  must  also  be 
thought  of  as  each  provides  a  certain  critical  skill  that  is  required  for 
sijccessful  plant  operation.    The  general  problem  posed  by  the  Ainiy  involves 
how  a  layaway  plant  can  be  started  up  as  quickly  as  possible  to  a  desired 
production  level.    The  measure  of  a  plant  s  ability  to  reach  the  production 

I  target  is  its  "readiness." 

The  particular  aspect  of  the  layaway  and  readiness  problems  addressed 
1  by  this  paper  encoopasses  the  use  of  design  techniques  that  minimize  re- 
j  liance  on  critical  skills  of  individuals  after  the  layaway  period.  The 
objective  of  providing  a  plant  design  so  self -documenting  that  its  oper- 
ation in  theory  and  practice  is  obvious  to  the  most  casual  observer  is 
both  impractical  and  xmrealizable .    However,  this  paper  is  intended  to  be 
j  both  evocative  and  provocative  so  that  at  least  the  impact  of  the  readi- 
i  ness  requirement  on  design  practice  can  be  explored  and  some  general  di- 
j  rections  indicated.    Toward  accomplishing  this  goal,  a  review  of  present 
'!  industry  attitudes  and  practices  toward  control  design  is  as  inportant  as 
I  proposed  solutions  for  the  future. 


i!  INDUSTRY  OMmVLITY 

The  Readiness  problem  (more  so  than  the  layaway  problem)  has  in- 
:  trigued  the  process  industry  in  general  for  many  years.    Taken  as  a  broad 
definition,  it  refers  to  a  plant  s  ability  to  move,  with  little  advance 
I  warning,  from  its  present  state  to  a  desired  state  as  quickly  and  ef fi- 
ll cently  as  possible.    Exanples  include:    seasonal  products  for  the  chemi- 
cal industry  (e.g.,  agricultural  chemicals,  antifreeze  additives);  raw 
material  changes  for  refineries  (e.g.,  crude  switching  control  (18));  and 
I  changes  in  plant  operations  caused  by  labor  strikes  or  walkouts.  The 
'  Readiness  problem  is  aggravated  by  projections  of  higiher  operator  and 
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staff  turnover  rates  and  a  general  reduction  in  the  experience  and  train 
ing  level  of  operating  personnel.    As  seen  by  these  exanples,  Readiness 
is  intimately  involved  in  the  my  a  plant  is  operated.    It  is,  therefore 
useful  to  review  some  of  the  operating  characteristics  of  plants  that  forn- 
a  set  of  axicois  \diich  vdll  always  mderlie  the  Readiness  issue. 


PLAOT  CHARACTERISTICS 

Figure  1  lists  a  series  of  characteristics  of  operating  plants  Which 
are  vrorth  considering.    With  many  control  loops  and  thousands  of  pieces 
of  mechanical  equipment,  a  large  plant  is  usually  alvjays  WDrking  with 
seme  automatic  control  loop  in  manual  or  some  pump  or  valve  down  and  out 
for  repair.    As  Bristol  (6)  points  out,  the  operation  of  a  large  process 
can  be  coopared  to  the  operation  of  an  old  car  operated  by  a  car  buff. 
At  any  point  in  time,  something  is  wrong.    But  the  operation  is  carried 
on  in  spite  of  all,  in  a  way  vMch  accannodates  the  current  problems.  If 
the  car  bums  oil,  oil  is  added  every  50  miles.    If  the  back  doors  are 
stuck,  everyone  piles  in  over  the  front  seats.    Likewise,   the  day-to-day 
operation  of  a  plant  is  also  subject  to  all  kinds  of  irregular  conditions 
or  constraints :    varaiation  in  raw  material  price  or  product  value ,  change! 
in  inventory,  or  vacations  of  first  run  operating  personnel. 

In  coping  with  such  situations,  it  should  be  noted  that  in  most 
plants  no  one  person  has  direct  responsibility  to  maintain  an  "on  control' 
condition.    The  instrument  department  fixes  "broken"  instruments;  the 
process  engineer  is  responsible  for  the  process  itself;  and  the  central 
engineer  is  too  far  away  fron  the  plant  for  day-to-day  contact  and  probabl 
wDuldn't  be  allowed  to  touch  the  control  system  anyway.    Yet  the  plant  rui 
and  a  semblance  of  Readiness  appears. 

Acceptance  of  advanced  control  technology  has  been  very  slow  in  the 
process  industries,  mainly  for  reasons  other  than  technology  (2).  A 
recent  Process  Industry  review  in  Science  (11)  pointed  out  that  "most 
observers  in  industry  agree  that  the  acceptance  of  advanced  technology  is 
far  less  than  the  state  of  the  art  would  permit."   Hence  technology  to 
improve  plant  Readiness  exists  and  more  will  be  developed,  but  acceptance 
problems  inherent  in  the  design  methodology  will  have  to  be  overcome . 

Finally,  it  should  be  recognized  that  control  equipment  maintenance 
and  operator  training  will  always  be  perceived  as  being  critical  even  if 
a  completely  automated,  reliable  plant  control  system  were  available. 
Such  a  system  is  just  not  considered  to  exist. 

The  previous  discussion  has  focused  on  the  plant  operation  from  a 
control  standpoint  because  the  control  system  design  actually  defines  the 
plant  operation  and  its  Readiness  potential. 
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OONTRQL  SYglEM  CHARACTERISTICS 


When  you  stop  and  t±LLnk  about  it,  almost  all  the  operating  procedures 
in  a  plant  are  based  on  some  aspect  of  the  control  system  design.  The 
steps  an  operator  goes  through  to  make  a  change  depends  on  the  layout  and 
design  of  the  control  panel.    The  control  loop  configurations  dictate 
allowable  control  variables  vMle  valve  sizing,  measurement  selection,  and 
sensor  placement  can  lead  to  processing  and  stability  problems.    As  a 
result,  the  person  \Ak)  ends       specifying  the  control  system  design  almost 
singlehandedly  specif  ices  the  operating  procedures  and  thereby  influences 
the  Readiness  posture  of  the  plant. 

One  wDuld  expect,  therefore,  that  such  an  important  Readiness  function 
as  control  system  design  would  be  of  prime  inportance  in  any  plant  design  - 
but  it  isn't.    Traditionally  and  across  industry  lines,  a  cost-effective 
plant  design  is  frozen  as  much  as  possible  once  detailed  engineering  is 
ready  for  field  construction.    Unfortunately,  control  system  decisions 
are  normally  carried  out  at  this  time  as  shown  in  Figure  2,  and  conse- 
quently, the  inpact  of  the  decisions  have  little  chance  to  feed  back  to 
the  process  design,  plant  operations,  and  Readiness  issues  vith  vMch  they 
usually  interact.    All  the  average  control  engineer  can  do  at  this  point 
is  design  conventional  control  schaues,  usually  based  on  the  individual 
plant  unit  operations.    These  are  then  hurriedly  conbined  into  some  form 
of  process  control  system  to  meet  plant  construction  and  start-up  sched- 
ules .    In  most  cases ,  these  time  schedules  do  not  allow  any  tailoring  or 
innovation  to  better  meet  Readiness  needs. 

Historically,  this  vrorked  out  reasonably  well  because  only  con- 
■  ventional  analog  hardware  systems  were  available  and  most  higher  level 
\  control  functions  were  carried  out  by  the  operating  personnel.  Hardware 
I  set  the  structure  and  ccoplexity  of  the  control  system.    Such  functions 
jj  as  anticipating  disturbances,   interacting  control,  and  changes  in  operat- 
\  ing  conditions  were  done  by  plant  operators. 

This  aspect  of  control  design  projects  along  with  scxne  other  inpor- 
tant  characteristics  are  given  in  Figure  3.    In  addition  to  the  lateness 
of  the  system  design  and  its  effect  on  plant  operations,  control  decisions 
tend  to  be  very  emotional  issues.    This  may  be  as  a  result  of  a  subcon- 
scious awareness  of  their  importance.    Decisions  for  a  small  control  con- 
puter  have  reportedly  reached  the  president's  office  in  some  ccmpanies 
vMle  more  expensive  process  equipment  purchases  (even  when  prior  ex- 
perience was  not  available)  have  been  handled  on  a  design  team,  level. 

The  result  is  that  one  must  recognize  that  control  systems  tend  to 
take  a  particularly  conservative  approach  and  are  not  siibject  to  any 
standard,  rigorous  design  procedures  as  is  fomd  for  sizing  equipment 
such  as  heat  exchangers  and  distillation  columns.    In  addition,  del^ation 
of  control  system  design  is  often  left  to  the  instrument  man,  vendor  or 
contractor  to  define. 
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To  a  great  extent,  however,  it  nust  be  acknowledged  that  the  processes 
to  be  controlled  are  analytically  very  cotnplex  so  that  even  a  sophisticated 
model  wDiold  only  approximate  reality  (6) .    Control  systems  are  indeed 
legitimately  designed  as  a  conpromise  between  xdiat  has  been  offered  by 
vendors,  \dnat  wDrked  in  the  past,  v^t  others  are  doing  and  vtet  side 
issues  are  present  at  any  particular  time.    There  is  no  better  vay. 
Dynamics  are  inportant  because  no  one  has  come  up  with  significant  im- 
provements over  linear  (or  nearly  linear)  feedback  control.    Even  feedback 
is  necessary  because  chanical  engineers  basically  don't  know  enough  about 
their  process  to  avoid  it. 

All  of  these  points  have  to  be  considered  \jhen  discussing  Readiness 
becaiase  they  inpact  on  possible  solutions  and  are  not  about  to  go  away. 
The  outlook  isn't  bleak,  however.    As  mentioned  previously,  this  system 
of  control  design  has  worked,  "rules  of  thumb"  have  been  established,  and 
plants  continue  to  operate  and  meet  objectives.    The  problem,  as  it 
effects  Readiness,  is  that  the  successful  operation  of  such  control  systems 
requires  a  level  of  technical  personnel  with  critical  skills  -  the 
Technocrat. 

THE  TECHNOCRAT 

The  loss  of  a  Technocrat  at  a  critical  mcment  can  be  devastating  to 
Readiness  because  of  the  loss  of  his  critical  skill. 

The  person' s  identity  is  not  always  obvious .    It  may  be  the  process 
engineer  who  picked  up  and  learned  the  little  procedures  that  really  make 
the  plant  run.    Or  the  FORTRAN  programnier  who  wrote  with  twists  of  logic 
known  only  to  him.    It  may  be  the  instinjment  man  \jhD  by  trial  and  error 
found  the  correct  grounding  wire  configuration  that  gave  stable  per- 
formance to  the  interface  system.    The  Technocrat  may  even  be  far  away 
from  the  plant,  tucked  in  a  research  and  development,  vendor,  or  con- 
tractor's office  vhere  his  design  notes  might  go  unnoticed,  ccoibined  with 
many  other  projects. 

Hidden  knowledge  is  the  weapon  of  the  Technocrat  \Mle  rapid  technol- 
ogical change  in  conpjters  and  control  equipment,  custcmized  installations, 
and  the  desirability  of  obtaining  a  high  level  of  total  plant  control 
provides  his  anmunition.    MDSt  Technocrats  don't  realize  lAhat  they  are 
or  recognize  their  potential  inportance  to  the  installation.    They  are 
well  intentioned  control  users  or  support  staff        go  imnoticed  until 
they  are  needed. 

Technocrats  can  not  be  eliminated.    A  completely  si:pplied  system  - 
customized  to  a  particular  application  -  vMch  doesn't  require  mainte- 
nance or  change  is  a  myth.    Such  system  responsibility  can  not  be  put 
aside  in  the  hope  it  will  not  be  needed.    There  is  no  secinrity  in  a  black 
box  I    Even  back-up  hardware  does  not  reduce  the  effect  of  the  Technocrat. 
The  back-iip  system  itself  requires  design,  maintenance  and  operating  know- 
how  to  make  it  an  effective  system. 
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is|         The  Readiness  level  for  a  mLmitions  plant  will  depend  on  how  well 
f '  the  effect  of  the  Technocrat  is  iiiLnimized  and  how  well  the  other  character- 
istics of  plant  operation  and  control  design  are  taken  into  account.  With 
j  that  as  a  background  let's  see  \^t  can  be  done. 


lESlQH  TECHNIQUES 

Analysis,  coranunication,  definition  and  enforconent  are  the  key  vrords 
that  describe  the  techniques  that  can  be  used  to  neutralize  the  effects 
of  the  Technocrat  and  the  present  control  system  design  methodology, 
thereby  relieving  the  future  critical  skill  requirements  and  increasing 
Readiness. 

The  techniques  to  be  discussed  do  not  come  without  a  price  and  trade- 
!  off s  will,  of  course,  be  necessary.    The  obvioias  trade-off  will  be  econcmic 
sji  since  discounted  cash  flow  puts  high  present  value  cost  on  a  plant  to  be 
j  used  in  the  future.    Less  obvious  is  the  trade-off  between  sinplicity  of 
i|  design  and  energy  conservation.    The  former  suggests  a  modular  design 
technique  (discussed  later) ,  while  the  latter  would  recomnend  a  tightly 
coordinated  plant  with  little  internal  hold-up  and  integrated  energy 
streams . 


1  DEFINITIONS 

i 

j         The  technique  of  establishing  a  consistent  set  of  definitions  is  espe- 
j  cially  critical  for  the  Hmitions  Production  Base  program  because  of  the 
j  many  different  organizations  involved  within  the  government  and  private 
i|  industry  as  veil.    Definitions  concerning  the  levels  of  autcmation  are 
'  very  critical  since  an  advanced  control  system  to  one  party  may  seem 
j  trivial  to  another. 
\ 

I         Attenpts  have  been  made  to  define  consistent  terminology  for  the  ele- 
I  ments  of  control  systems .    Most  notable  is  the  International  Electro- 
;;  technical  Cotmiission  standard  currently  being  reviewed.    But  this  is  not 
I  enough.    An  extension  to  the  definitions  of  individual  control  elements 

is  needed  to  adequately  disciass  control  system  design  and  possible 
;  hierarchies.    It  has  been  proposed  that  an  adequate  description  of  ad- 
[  vanced  control  systems  requires  at  least  tvvo  distinct  dimensions  (2)  as 

shown  in  Figure  4. 

lii         Ocie  axis  represents  increasing  levels  of  control  autonation  (involv- 
li  i  ing  the  physical  operation  and  control  of  the  plant  or  process) ,  and  the 
[jj!  other  represents  increasing  levels  of  process  management  (yihere  process 
'!  management  is  defined  as  the  gathering  and  application  of  both  process 

and  business  information  for  decision  making) .  As  higher  levels  of  con- 
[  trol  autctnation  are  inplemented,  increasing  numbers  of  measurements  must 

be  brought  into  the  control  system.    Furthermore,  such  a  system  must  also 
III  encompass  larger  segments  of  the  plant  because  of  the  need  for  multiunit 
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coordination.  The  same  measurements  needed  for  advanced  autctnation  are 
also  useful  in  decision  making,  and  as  one  automates  the  control  of  the 
plant,  it  also  makes  sense  to  incorporate  more  information  gathering  and 
application  functions  in  the  overall  control  syston.    Therefore,  most  ad- 
vanced control  systems  will  usually  progress  along  a  trajectory  as  shown 
in  Figure  4,  where  control  automation  and  process  management  functions  in- 
crease simultaneously. 

There  is  insufficient  time  to  go  into  the  detailed  description  of 
each  level  in  this  paper,  but  the  major  characteristics  are  surnnarized  in 
Tables  1  and  2. 

Understanding  the  use  of  these  definitions  is  important.    They  are 
meant  to  convey  a  "sense"  of  where  a  plant  fits  in  the  definitional 
structure.    The  scale  is  continuous  but  one  Mxild  find  it  a  fruitless 
experience  to  methodically  count  the  number  of  different  types  of  con- 
trol he  has  in  a  particular  plant  and  relate  that  to  control  level.  In- 
stead a  user  should  read  the  coqplete  definitions,  consider  them,  and 
then  think  of  the  operation  as  a  \dnole  and  pick  the  level  of  control  or 
management  that  seems  appropriate. 

These  definitions  are  reported  to  be  a  good  base  and  a  valuable  tool 
to  developing  consistency  in  discussions  concerning  the  level  of  process 
management  and  control  (2) .    Such  consistency  is  essential  to  good  coEcmu- 
nications  between  government  agencies,  vendor  and  industrial  connLinities , 
especially  in  these  times  of  rapidly  changing  control  technology. 

Equally  as  inportant  as  the  control  definitions  are  the  definitions 
and  responsibilities  of  those  groups  taking  part  in  the  design,  con- 
struction and  operations  of  the  munition  based  plants.    A  sanple  set  of 
participants  and  responsibilities  is  listed  in  Figure  5.    The  list  is  a 
suggested  one  but  the  main  point  is  that  the  list  must  be  formulated  and 
the  responsibilities  detailed. 

Consistent  definitions  (probably  made  up  by  the  government  and 
operating  sectors) ,  if  adhered  to,  will  sinplify  decision  making  and  cause 
it  to  be  based  on  a  visual  set  of  objectives. 

ITERATIVE  DESIGN  PROCEDURE 

Many  factors  affect  the  design  of  a  modem  process  control  system. 
The  process  itself  is  the  dominant  influence  since  it  defines  ^/^hat  actions 
must  be  taken  for  proper  operation.    Parameters  of  the  control  system 
\\Mch  carry  out  these  actions  are  determined  by  hardware,  software, 
measurement,  control,  and  comnunication  technologies  as  well  as  hunan 
factors  (Figure  6) .    All  of  these  aspects  involve  the  critical  skills  of 
many  people.    However,  we  noted  previously  that  there  is  presently  little 
interaction  between  the  design  and  control  engineers.    Consequently  a 
change  must  take  place  afid  a  ne^  technique  must  be  employed. 
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A  recannended  plant  design  technique  is  shown  in  Figure  7  which  allows 
for  an  integration  of  process,  plant  operation,  and  control  goals  early 
in  the  plant  design  -when  a  free  exchange  of  concepts  can  be  acconplished. 
Within  this  structure,  questions  concerning  process  requirements,  plant 
operation,  and  process  economic  sensitivity  as  well  as  the  technical  areas 
of  measurement,  control,  ccmiiunications ,  and  degree  of  control  distribution 
can  be  considered.    These  and  other  factors  to  be  considered  during  this 
design  period  are  listed  in  Figure  8.    Thus  the  process  goals  and  plant 
operating  goals  are  considered  together  by  the  design  and  control  engineers 
in  an  iterative  fashion. 

Oae  of  the  biggest  problems  in  examining  a  plant  design  by  the  tech- 
nique described  above  is  the  lack  of  formal  methods  of  process  syston  de- 
composition.   Such  decoEposition  leads  to  smaller  sub- systems  which  are 
easier  to  analyze.    Bernard  (5)  suggests  a  method  \^ch  involves  a 
horizontal  break-dcwn  based  on  the  concept  of  "operating  units"  instead 
of  the  standard  unit  operations.    The  result  is  a  structure  with  many 
modes  of  minimum  interaction.    Carefully  defined  purposes,  goals,  and 
broad  functional  requirements  are  basic  to  the  structure.    Ihe  key  to  a 
good  analysis  is  the  willingness  to  write  down  pmrposes  for  different 
parts  of  the  system  and  functions  that  seem  obvious  and  simple.    A  good 
measure  of  the  quality  of  the  structuring  is  how  little  the  \ih.ole  system 
must  be  changed  to  accomnodate  changes  in  any  of  its  parts. 

From  a  Readiness  viewpoint,  these  design  techniques  bring  many  crit- 
ical skills  together  at  one  time,  broadens  tftie  base  of  people  who  were 
involved  with  the  design  discussions,  and  at  least  presents  a  better 
opportunity  to  document  the  discussions  for  later  use. 


ADVANCED  CCMPUTING  SYSTEMS 

The  use  of  advanced  computing  systems  for  the  analysis  of  process 
flow  sheets  is  an  extremely  valuable  technique  for  development  of  sctne 
phases  of  the  munitions  program.    The  computing  systems  make  extensive 
use  of  the  most  understood  process  characteristics  -  the  conservation 
laws:    material  balance,  compcnent  balance,  and  energy  balance.    They  also 
provide  a  solid  base  for  design  documentation  which  can  be  easily  and 
systematically  reviewed  at  a  later  date. 

Many  of  the  systems  available  today  were  developed  since  the  late 
1960 's  and  are  caisidered  a  "second  generation"  (10).    Exanples  are  Mon- 
sants's  FLCWTRAN,  Exxon's  COPE,  Union  Carbide's  IPES,  and  DuPont's  CPES. 
In  general,  these  programs  carry  out  steady  state  process  s inflations  in- 
volving streams  of  vapors  and  liquids  including  heat  and  material  balances . 
For  large  plants  most  engineers  would  not  consider  manually  calculating 
design  equations  for  distillation  colums,  heat  exciiangers,  etc.  The 
main  problem  with  these  programs  is  that  they  are  not  universally  available 
and  require  specific  skills  to  use. 
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MIT  has  recently  been  awarded  a  t±iree  million  dollar  grant  by  ERDA. 
to  develop  a  3rd  generation  computer-aided  design  language  (12) .  The 
project,  named  ASPEN,  will  concentrate  on  extending  existing  programs 
to  include  fossil  energy  conversion  processes.    In  addition,  it  is  hoped 
that  ASPEN  will  be  accepted  as  a  universal  system  \ised  by  all  process 
engineers  and  taught  to  students  at  universities.    High  priority  is  being 
given  to  make  ASPEN  readily  accessible,  fully  documented,  easy  to  use,  and 
able  to  be  extended  to  include  dynamic  simulations. 

A  3rd  generation  flow  sheeting  system  such  as  ASPEN  would  provide  a 
nonpersonal  link  that  could  be  used  in  the  research,  development,  pilot 
plant,  engineering  and  plant  performance  evaluation  states  of  a  project. 
A  word  of  warning,  though:    assurrptions  inherent  in  any  simulation  should 
be  clearly  documented  and  be  available  when  the  program  is  used  again 
after  a  layaway  period. 


MAN-MACHINE  INTERFACE  TECHNIQUES 

Techniques  for  designing  time- independent  man-machine  interfaces  are 
only  beginning  to  be  thought  about  and  are  certainly  not  fully  developed. 
A  general  methodology  is  required,  however,  as  increasing  levels  of 
autcmation  are  resulting  in  a  nuuiber  of  the  human  operator's  mental  pro- 
cesses being  taken  over  or  augmented  by  information  processing  systems. 

Looking  back  one  sees  that  the  operator /process  system  was  tightly 
coupled.    The  operator  was  "in  touch"  with  his  environment;  he  determined 
the  operating  procedure;  and  his  "experience"  was  a  key  to  the  successful 
operation  of  the  plant.    As  size  and  coiplexity  of  processes  increased, 
more  efficient  methods  for  monitoring  and  control  were  required  resulting 
in  the  central  control  room  with  its  large  panel  of  instrunents . 

The  panel  design  and  display  format  are  traditionally  developed  based 
on  the  designer's  "experience"  in  understanding  the  role  of  the  operator 
and  his  needs .    Infrequently  some  human  factor  may  be  added  TA^ien  the 
selection  of  colors  or  positioning  of  piishbuttons  is  involved.  In 
general,  the  operator  display  -  even  more  so  than  the  control  system  - 
is  almost  included  as  an  afterthought  to  the  design  of  the  plant.  Con- 
sequently, this  usually  means  a  study  of  human  operator  behavior  and  a 
definition  of  his  task  was  never  included  in  the  early  planning  of  the 
installation.    From  a  Readiness  stanc^oint,  this  means  that  critical 
operator  skills  may  not  be  recognized  mtil  after  the  plant  start-up  is 
initiated. 

Techniques  to  counteract  this  problon  are  b^inning  to  develop. 
Basically  they  involve  analyzing  the  task  required  of  the  human  sifl)ervisor 
as  he  watches  the  control  system  as  shown  in  Figure  9.    The  tasks  can  be 
suroraarized  as  follows: 

Learning,  understanding,  and  interpreting  goals  required  of  him. 
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I  MDnitxjring  t±ie  system  outputs  so  that  from  the  control  actions  he 

I  can  identify  the  dynamics  of  the  system. 

Planning  and  determining  v^ch  control  actions  should  be  performed. 

Inputting  the  appropriate  data  to  the  control  system  for  both 
initialization  and  on-line  adjustments. 

Intervening  in  order  to  switch  from  manual  to  supervisory  control . 

The  guidelines  for  Man-Machine  Interface  Design  established  by  the 
International  Purdue  Workshop  (14)  provide  an  excellent  base  for  defining 
the  operator  tasks  and  relating  them  to  display  design.    It  WDuld  be 
inpossible  to  design  such  a  flexible  display  that  would  meet  unknown  re- 
quirements 10  years  from  now,    A  better  technique  is  to  specify  details 
i  about  the  operator,  his  task,  and  environment  as  shown  in  Figure  10  and 
then  design  the  interface  accordingly. 

I  For  exanple:    The  education  level  and  prior  plant  esqjerience  of  the 

j  operator  should  be  specified  as  well  as  vtether  training  will  be  carried 
j  out  away  from  the  plant  or  covered  on-the-job  by  the  display  system.  The 
'  job  requirements  also  have  to  be  detailed  including  the  type  of  decisions 
I  the  operator  will  be  expected  to  make,  the  degree  of  supervision  that  will 
'  be  provided,  and  the  use  the  operator  is  expected  to  make  of  the  instru- 
ments (sinple  monitoring,  cotrparison  of  set  points,  etc.).    Even  details 
j  such  as  the  location  of  the  control  roan  (near  or  far  frctn  the  equipment) 
i  or  its  size  (^11  the  operator  be  standing  or  seated)  will  effect  the 
display  design. 

I  Basically  the  operator  performs  two  functions:    (1)  getting  informa- 

tion from  the  system;  and  (2)  taking  action  on  the  control  elements.  The 
|j  operator's  need  for  information  may  be  continuous,  periodic,  on-demand  or 
t  automatic ;  and  his  choices  of  action  varies  widely  in  nuciber  and  type ,  de- 
j  pending  ipon  process  conditions  and  the  control  elements  with  vMch  he  is 
j  dealing. 

i         A  present  answer  for  this  situation  is  an  operating  procedure  match- 
I  ing  available  actions  to  a  particular  task  and  the  current  conditions, 

with  a  variable  function  keyboard,  vMch  changes  its  function  under  program 
i  control  (15) .    A  future  answer  may  be  indicated  by  a  stucfy  underway  at 

Delft  University,  the  Netherlands,  to  model  the  huuan  operator  in  order  to 
!j  analyze  his  ability  to  react  to  situations  of  excessive  data,  emergency 
'  conditions  and  analysis  of  information  (4) . 


ANALYSIS  TECH^QUES 

Fault  tree  analysis  is  a  descipline  ^^ch  attenpts  to  model  and 
analyze  failure  processes  of  engineering  systems.    Construction  of  the 
fault  tree,  a  basic  step  in  fault  tree  analysis,  requires  an  intimate 
knowledge  of  the  manner  in  which  a  system  is  designed  and  operated. 
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Lanibert  (16)  showed  how  t±ie  technique  could  be  used  to  optimally  locate 
sensors  withia  the  chemical  processing  systems. 

A  basic  problem  vdth  fault  tree  analysis  is  the  time  required.  A 
cooplete  flow  sheet  of  the  process  is  needed  and  all  the  hazardous  events 
that  might  occur  must  be  documented.    Each  potentially  hazardous  event 
may  then  be  evaluated  to  determine  the  possible  paths  t±at  led  to  that 
event.    These  sets  must  contain  "all"  the  inportant  events  in  the  proper 
logical  relationships.    A  paper  by  Powers  and  Lapp  (17)  discusses  a 
conputer- aided  fault  tree  synthesis  procedure.    They  estimate  that  each 
fault  tree  will  often  require  tvfo  to  three  man-days  to  carry  out  the 
generation,  documentation,  and  coiputation  on  the  analysis.    They  also 
point  to  a  25  man-year  effort  in  a  U.S.  Atomic  Energy  Cccmission  report 
that  was  required  to  generate  fault  trees  for  one  boiling  water  reaction 
and  one  pressurized  water  reactor. 

Fault  tree  analysis  does  provide  a  means  to  utilize  the  critical 
skills  of  the  designers  at  one  point  in  time  and  logically  document  the 
results  for  use  later. 


CmrBOL  TECHNIQUES 

We  can  probably  well  assume  that  the  time  and  manpower  requirements , 
in  addition  to  a  lack  of  process  knowledge,  place  a  practical  limit  on  the 
use  of  fault  tree  analysis.    Therefore,  a  different  kind  of  technique  is 
needed,  a  technique  xdiich  is  based  on  operation  rules.    The  tuning  of 
controllers,  the  diagnosis  of  alarm  conditions  and  the  analysis  of  process 
data  are  cairrried  out  by  operations  oriented  personnel  xvho  understand  and 
use  the  process  criteria.    Unfortunately,  this  means  reliance  on  the 
critical  skills  of  these  individuals. 

Adaptive  control  techniques  is  one  means  of  overccming  these  problems. 
The  problem  in  designing  adaptive  control  systems  has  been  to  come  up  with 
a  scheme  ^lich  includes  a  minimum  amount  of  coiputation,  requires  little 
special  mder standing,  and  which  will  adapt  most  processes  under  as  many 
disturbance  situations  as  possible.    Such  a  system,  based  on  a  simple 
pattern  recognition  technique  has  been  proposed  (7 ,  8) .    The  technique 
matches  the  way  an  operator  tunes  -  to  pattern  or  shapes  rather  than 
particular  response  values. 

Adapation  will  be  an  essential  future  characteristic  of  the  fully 
autcoiated  plant.    With  it  hunan  interference  will  not  normally  be  necessary 
in  any  loop  except  for  an  initial  pattern  specification.    In  addition,  a 
pattern  based  adaptation  can  prevent  instability  from  ever  occurring  in 
any  loop. 

Pattern  recognition  can  also  be  also  be  put  to  good  use  as  a  part  of 
a  system  to  help  diagnose  plant  problems  after  an  alarm  condition  and  to 
help  predict  vhen  an  alarm  condition  is  about  to  occur  (9) .    Usually  flash- 
ing alarms  present  only  the  "symptoms"  of  the  malfunctioning  process  and 
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responsibility  for  finding  the  cause  and  determining  corrective  action 
[  is  left  to  the  operator .    The  establishment  of  a  synptcm  pattern  is  simpler 
than  the  fault  tree  described  earlier  as  only  the  basic  paths  are  followed 
i  for  the  most  important  faults .    From  that  point  the  system  stores  new 
patterns  each  time  an  alarm  is  sounded.    The  operator  may  also  enter  new 
patterns.    The  procedure  is  then  one  of  matchirig  the  actual  alarm  history^ 
to  the  dictionary.    Matching  of  actual  symptom  pattern  will  alhiw  diagnosis 
of  the  fault.  Other    "teachable"  alarm  and  disturbance  analysis  system 
have  also  been  suggested  (13) . 

Newer  systems  produced  under  the  name  of  "model  reference"  control 
systems  are  aimed  to  help  the  operator  assimilate  data  frctn  infrequent 
laboratory  sanples  vMch  may  have  randctn  errors .    The  technique  uses  a 
model  to  take  control  action  but  iqjdates  the  model  parameters  frcm  an 
analysis  of  the  laboratory  sanples  using  a  Kalman  filter  approach  (6) . 
j  The  system  can  operate  through  temporary  measurement  failures  without 
suddenly   degrading  control  performance. 

I        Of  course,  designing  the  plant  to  be  inherently  stable  in  the  first 
place  eliminates  many  of  the  control  problems  and  is  not  as  inpossible  as 
one  might  imagine.    Tanks  with  overflows  for  normal  operation  require  no 
level  controller.    A  very  large  heat  exchanger  and  a  vast  constant  tem- 
perature  bath  could  be  used  to  keep  temperature  control  in  a  stable 
1  region  -  capital  cost  would  be  enormous ,  however .    While  sane  self - 
i  regulating  control  techniques  can  be  used,  the  majority  of  the  control 
I  will  involve  sane  measure  of  cooplexity  and  interdependence . 
I 

I         In  the  past,  if  the  operator  was  uncertain  about  a  loop,  he  put  it  in 
'  manual  operation.    With  any  degree  of  coup  lex  loops,  putting  a  loop  on 
1  manual  might  destabilize  the  process  or  otherwise  affect  the  working  or 

safety  of  other  loops.    Hence,  use  of  advanced  control  techniques  should 
!!  include  an  interlock  between  the  auto/manual  functions  and  also  provide  a 
I  range  of  operational  states  between  the  conventional  manual  operation  and 
I  the  full  automatic. 


OTHER  TECHNK^JES 

Sane  other  techniques  are  worth  coimenting  on  as  they  can  affect  the 
critical  skill  requirement  problen  y/jhesn.  Readiness  is  required. 

■         A  training  similator  developed  along  with  the  plant  design  and  verified 
[  during  the  initial  plant  tests  can  be  used  to  train  operators  in  the  future 
j  or  maintain  personnel  with  critical  skills  during  the  layaway  period. 

Such  a  simulator  can  easily  grow  out  of  proportion  and  in  excess  of  its 
1  value.    Only  the  critical  sections  of  the  plant  need  be  similated  and  the 
'  simulation  level  should  reflect  the  operating  characteristics  rather  than 
I  be  an  exact  mathematical  representation  of  the  process.    A  movie  or  video 

tape  would  provide  good  documentation  of  the  way  the  plant  should  be  con- 
j  trolled. 

I 
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Good  documentation  is  essential  for  technology  transfer  at  some 
future  time.    Techniques  enployed  here  include:    (1)  listing  assinptions 
used  during  the  design  phase;  (2)  visibility  given  to  validity  tests  of 
the  design  programs  during  plant  start-up  to  instill  confidence;  (3)  use 
of  structured  languages  for  computer  programning  to  facilitate  reading; 
and  (4)  the  use  of  hybrid  flow  chart  type  languages  for  showing  both  the 
parallel  activity  or  simultaneous  logical  tests  inherent  in  sequential 
operations . 


SIMIARY 

Foxjr  conclusions  stand  out  as  necessary  to  increase  the  possibility 
of  obtaining  a  time  invariant  process  design  vhich  possesses  excellent 
Readiness  characteristics  because  of  a  inininial  depeiidence  on  future  crit- 
ical skills: 

(1)  tfeintain  a  sinplicity  in  design  through  modular  design  techniques 
making  use  of  a  systems  approach  and  incorporating  control  early  j 
in  the  planning  stage.  i 

(2)  Place  emphasis  of  making  sure  the  lowest  level  of  control  (single 
loop  temperature,  pressure,  flow,  etc.)  is  vrorking  properly  be-  | 
cause  all  other  techniqiies  will  assume  that  this  has  been  i  j' 
acconplished. 

(3)  Allow  for  the  simultaneous  evolution  of  information  and  design 
vMch  will  inevitably  take  place  since  at  any  point  in  the 
design  process  knowledge  will  always  be  lacking. 

(4)  Decide  viio  will  enforce  the  control  and  documentation  procedures 
agreed  upon. 
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PLANTS  RUN  FRCM  MINI  DISASTER  TO  MINI  DISASTER 
OLD  CAR  SYMMM: 

NO  RESPONSIBILITY  FOR  "ON  CONTROL"  CONDITION 
LITTLE  CHANGE  IN  OPERATION  CAUSED  BY  TECHNOLOGY 
CONTROL  MAINTENANCE  PROBLEMS 
OPERATOR  TRAINING 


FIGURE  1 
PLANT  OPERATING  QIARACTERISTICS 
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PROCESS  GOALS 


OVERALL  DESIGN 

I 

SPECIFIC  AREA  mSIGN 

I 

PROCESS  EVALUATION 

i 

DFTATTKD  ENGINEERING 

i 

COTIRDL  EQUIPMENI  SELECIION 

i 

PLANT  CONSTRUCTIOJ  EESIGN 

^  FROZEN 
START-UP 


OPERATING  PLANI 


FIGURE  2 
DESIGN  APPROACH  -  TYPICAL 
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FINAL  STEP  IN  PLANI  ENGINEERING 
EMDTIQNAL  ISSUE 
EFFECTS  PLANT  OPERATION 
USUALLY  CONSERVATIVE  APPROACH 
OFTEN  IZFT  TO  VENDDR  TO  DEFINE 
NO  STANDARDIZED  APPROACH 


FIGURE  3 
OKTROL  PROJECT  CHARACTERISTICS 
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NOIiVWOiLnV  lOHUiOO  AO  SIIMI  OMISVaMOMI 
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TABLE  lA 

FUNCTIONAL  CHARACTERISTICS  OF  CONTROL  TECHNOLOGIES  REQUIRED 


^ — ^..^^^^     INCRFASING  LEVELS  OF 

CONVEN- 

SEMI- 

FULLY 

^ — ..CONTROL  AUTOMATION 

MANUAL 

TIONAL 

ADVANCED 

AUTO. 

AUTO. 

FUNCTIONAL                   -^^.^^^  DESIRED 

OPERA.TION 

OPERATION 

OPERATION 

Ur  r.A>i  I  iUii 

CHARACTERISTICS  ^""""-^.^ 

OF  COOTROL  TFCHNOLOGIFS  REOUIRED^-~~>.^ 

MEASUREMENTS 

Basic  Measurements  (T,  P,  L.  F) 

X 

X 

X 

X 

X 

Advanced  (Conposl tion ,  Vibration) 

X 

X 

X 

Conrlex  (Product  Quality) 

X 

X 

CONTROL  DECTSTONS 

On-Off  Controller 

X 

X 

X 

X 

j-Moue  LontroiXer 

X 

X 

X 

X 

Model  Based  Controller 

X 

X 

X 

Alarm  Dlaf^nostic  Routines 

X 

X 

X 

Automatic  Controller  Tuning  Routine 

X 

X 

X 

DISPLAY  AND  REPORTING 

Display  and  Recording  of  Variables 

X 

X 

X 

X 

NA  For 

Annunciate  Alams 

X 

X 

X 

Normal 

Alann  Diagnostic  Messages 

X 

X 

Operation 

CONTROL  MODELS 

Simple  Heat  k  Mat.  Balance  Models 

X 

X 

X 

Dynamic  Process  Models 

X 

X 

Detailed  Safety  Models 

X 

X 

VARIABLE  MANIPULATION 

Actuators  (Basic) 

X 

X 

X 

X 

X 

Actuators  (Increased  Accuracy) 

X 

X 

Actuators  (Increased  Reliability) 

X 
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TABI£  IIA 

FUNCTIONAL  CHARACTERISTICS  OF  PROCESS 
MANAGEMENI  TECHNOLOGIES  REQUIRED 


^^"""-".....^^^^     INCREASING  LEVELS  OF 
^■~^-~-,^OCESS  MANAGEMENT 
FUNCTIONAL  DESIRED 

CHARACTERISTICS  OF   

FROCEPS  MCT.  TECHNOLOGIES  REO'JIRED'*--^ 

RUDIMEN- 
TARY 

CONVEN- 
TIONAL 

ADVANCED 

SEMI- 
AUTOMATIC 

AUTOMATIC 

INFORMATION  GATHERING  AND  CONVERSION 

Process  Data  Gatherinj^ 

Process  Data  Entry 

Business  Data  Gathering  And  Entry 

X 

X 
X 

X 
X 
X 

X 
X 
X 

X 
X 
X 

DECISION  ANALYSTS  TECHNIQUES 

Material  And  Energy  Balance 
Computation  Of  Performance  Variables 
Exception  Identification 
Inventory  Analysis 
Scheduling 

Optimization  Algorithms 
Forecast  Systcns 
Process  Diagnostics 

X 
X 
X 

X 
X 
X 
X 
X 
X 

X 
X 
X 
X 
X 
X 
X 

X 
X 
X 
X 
X 
X 
X 

DISPLAY  AND  REPORTING 

Data  Logp'ing 

Trend  Displays 

Exception  Reporting 

Receipts,  Orders,  Shipments  Display 

Pattern  Displays 

User-Directed  Hierarchical  Display 

X 

X 
X 
X 

X 
X 
X 
X 
X 
X 

X 
X 
X 
X 
X 
X 

NA 
Fcr 

Normal 
Operation 

DECISION  MODELS 

Saftey  and  Environmental  Model 
Material  Balance  Model 
Energy  Balance  Model 
Inventory  Model  (e.g.  EOQ) 
Scheduling  Model 

Optimization  Model  (e.g.  LP,  NLP) 
Distribution  i  Transportation  Model 
Forecast  Model  (e.g.  Econometric) 
Dinirnostlc  Model 

X 
X 
X 

X 
X 
X 
X 
X 
X 
X 

X 

X 
X 
X 
X 
X 
X 
X 
X 

X 
X 
X 
X 
X 
X 
X 
X 
X 

PROCESS  MANAOE^^ENT  DIRECTIVES 

Operating  Guides 

Plant  Operating  Conditions 

Raw  Material  Selection 

Shipping  tc  Distribut'n  Instructions 

Utilities  Manp-Kement 

Maintenance  Schedules 

Raw  Material  Ordering 

X 

X 
X 
X 
X 
X 
X 

X 
X 
X 
X 
X 
X 
X 

X 
X 
X 
X 
X 
X 
X 
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U.  S.  ARMY 


-    BASIC  PRODUCT  REQUIREMENTS 
PRIORITIES 
FUNDING 


OPERATING  COMPANIES    -    OPERATIONAL  (FUNCTIONAL)  REQUIREMENTS 
PROCESS  TEQINOLOGY 
SCHEDULE 

PROJECT  MANAGEMENT 

REALIZABLE  SYSTEM  SPECIFICATIONS 

CONTRACTORS  -  CajSTRUCTION 

INSTALL/^TION 

VENDORS  -    UNDERSTAND  USER  NEEDS 

PROVIDE  VIABLE  TECHNOLOGY  OPTIONS 

lESIGN  CONTROL  SYSTEt4  TO  MEET  FUNCTICNAL 
REQUIREMENTS 

INSTALLATION  AND  CCM^SSIONING  SUPERVISION 

MEET  CONTROL  SPECIFICATIONS 


FIGURE  5 

PARTICIPANTS  AND  RESPONSIBILITIES 
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PROCESS  GOALS    <  PROCESS  OPERATION 

OVERALL  DESIGN  CCNTRDL  EESIGN 


SPEanC  AREA  EESIGN 

i 

EVALUATION 

i 

IKIAILED  ENGINEERING 

i  — T- 

PLANT  oajjSTRUcria^  I 

I  raSIGN 
START-UP  ™^ 


OPERATING  PLANT 


;  FIGURE  7 
DESIGN  APPROACH  -  RECatlENDED 
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PLANT  GOALS 

PLANT  OPERAnONAL/MANAjGEbfNT 

PHILOSOPHY 
STAFFING  REQUIREMENTS 
MAN/MACHINE  FUNCTICN  lESCRIPTION 
CCNTROL  RDCM  PROCEDURES 
COMMUNICATIONS  NEEDS 
AREAS  FOR  CCKTROL  ENFORCEMENT 


SAFETY  CONSIDERATIONS 

STANDARD  OPERATING  PROCEDURES 

PROCESS  EOONCKLC  SENSITIVITY 

CONTROL  STRATEGY 

LEVEL  OF  CONTROL  DISTRIBUTION 

PROCESS  EQUIPMENT  REVIEW 

MEASUREMENT  NEEDS/MEANS 

(XmEDL  SYSTEM  DESIGN 
RESPONSIBILITIES 


FIGURE  8 

FACTORS  OQNSIIERED  DURING  SYSTEM  lESim 
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THE  OPERATOR: 


EDUCATION 

OPERATIOSI  EXPERIENCE 
TRAINING  LEVEL 
MENIAL  LOAD  LIMIT 

THE  TASK; 

JOB  REQUIREMENIS ' 
DECISION  LEVEL 
DEGREE  OF  SIFERVISICN 

INSTRUMENI  USE 

THE  ENVIRONMENT: 

LOGATIC»^ 
SIZE 

PHYSICAL  REQUIREMENTS 


FIGURE  10 
WHAT  THEY  SPECIFY 
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APPENDIX  I 


LIST  OF  PAKTICIPANTS 

WDRKSHDP  ON  COTTROL  SYSTEMS  READINESS 
FOR  MUNITIONS  PLANIS 
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LIST  OF  PARTICIPANTS 
WORKSHOP  ON  CONTROL  SYSTEMS  READINESS 
FOR  MUNITIONS  PLANTS 
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AGENDA  AND  TECHNICAL  PROGRAM 


WORKSHOP  ON  CONTROL  SYSTEMS  READINESS 
FOR  MUNITIONS  PLANTS :    A  FIRST  PASS 


Room  322 
Stewart  Center 
Pijrdue  University 
West  Lafayette,  Indiana  47907 


Septenber  19-20,  1977 


Manday,  September  19,  1977 
SESSION  NUMBER  1 

Chairman:    Mr.  Doiiglas  A.  MDrlock 
U.S.  Ann7  -  DROM-PEM-T 
Project  Manager  for  Munitions 
Production  Base  Modernization 
Dover,  New  Jersey  07801 

AM 

09:00  -  09:30   Introduction  Remarks 

Mr.  Douglas  A.  Morlock 

Lt.  Col.  Denis  C.  Dice 
U.S.  Arny  -  I^CPM-PBM-T 
Production  Base  Modernization 
Dover,  New  Jersey  07801 

and 

Dr.  John  M.  Evans,  Jr. 
Manager,  Automation  &  Control  Program 
National  Bureau  of  Standards 
Washington,  D.C.  20234 
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09:30  -  10:00     Past  Experience  in  the  Reactivation 
of  Army  Ainnunition  Plants 

Mr.  Augie  J.  Zahatko 

Chief  of  Production  Base  Concepts  Office  AFRCCM 
Headquarters,  U.S.  Array  Armanient  Material 
Readiness  Comnand  DSAR-INC 
Rock  Island,  Illinois  61201 


10:00  -  10:15     Coffee  Break 


10:15  -  10:30     Current  Procedures  and  Problems  in  the  Layaway 
and  Reactivation  of  Process  Control  Systenis 

Mr.  Ferenc  T.  Beiwel 
Mechanical  Engineer 

U.S.  Army  Industrial  Base  Engineering  Activity 
Rock  Island,  Illinois  61201 


10:30  -  10:45     Policies  &  Procedures  for  Layaway  Equipment 
Management  6c  I'laintenance 

Mr.  John  Nemanich 

Industrial  Specialist 

Headquarters,  U.S.  Army  Armament 

Readiness  Carmand 

Rock  Island,  Illinois  61201 

10:45-11:15     An  Overview  of  the  Performance  Characteristics 
of  Sensory  Transducers 

Mr.  Paul  S.  Lederer 

Chief,  Components  and  Application  Section 
Institute  for  Applied  Technology 
National  Bureau  of  Standards 
Washington,  D.  C.  20234 


11:15  -  12:15     Discussion  of  I'laterial  of  Morning  Reports 


m 

12:00  -  01:30  Lunch 
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SESSION  NUMBER  2 


Chairman:    Dr.  Gordon  J.  VanderBrug 

Project  Manager,  Aatomtion  &  Control  Program 
National  Bureau  of  Standards 
Washington,  D.  C.  20234 


01:30  -  01:40  Remarks 


01:40  -  02:30     Layaway,  Standby  and  Reactivation  Procedures 
for  Computer  Magnetic  Media 

Mr.  Sidney  B.  Geller 

Manager,  Magnetic  Media  Group 

Institute  for  Conpater  Science  5c  Technology 

National  Bureau  of  Standards 

Washington,  D.  C.  20234 


02:30  -  02:40     Coffee  Break 


02:40  -  03:10     Environmental  Effects  on  Electrical 

and  Electronic  Devices  and  Equipment  Under 
Long  Storage  Conditions 

Mr.  Albert  J.  Graf 

Reliability  Engineer,  Large  Caliber 

Weapons  Systems  Laboratory 

U.S.  Army  Armament  Research  &  Development 

Ccraiiand  DRDAR-LCA 

Applied  Sciences  Division 

Dover,  New  Jersey  07801 


03:10  -  03:45     Layaway  Program  for  Control  Valves 

Mr.  Joseph  Van  Damme 
Principal  Development  Engineer 
The  Foxboro  Ccrapany 
Foxboro,  Mass.  02035 
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03:45  -  04:10     The  Independent  Testing  and  Proving  of 
Hardware  and  Software  Elements  of  Plant 
Computer  Control  Systems  on  Reactivation 
of  ly^jnitions  Plants 

Mr.  Dale  Zobrist 

ELDEC  Corporation 

P.  0.  Box  100 

LynwDod,  Washington  98036 


04:10  -  04:25     The  Layaway  and  Reactivation  Problan 
of  Beet  Sugar  Mills  for  Their  Annual 
Canpaigns  Including  Management,  Manpower 
and  Training  Aspects 

Professor  Theodore  J.  Williams 

Purdue  Laboratory  for  Applied  Industrial  Control 

Schools  of  Engineering 

Purdue  University 

West  Lafayette,  Indiana  47907 

with  the  help  of 

Mr.  Daniel  D.  Lesser 
Factory  Manager 
Northern  Ohio  Sugar  Cocpany 
Findlay,  Ohio  45840 


04:25  -  06:00     Discussion  of  the  Topics  of  the 
Afternoon  Reports 


BAIIQUET  AND  EVENINS  SESSION 


Private  Banquet  Roan 
Morris  Bryant  Restaurant 


07:00  -  08:00  Dinner 
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08:00  -  10:00     Discussion  of  the  Program  of  the 

U.S.  Army  for  Mmitions  Production 
Base  Modernization  and  Expansion 

Dr.  Gordon  J.  VanderBrug 
National  Bureau  of  Standards 

Ilr.  Carl  Beaulieu 

and 

Mr.  Douglas  A.  Mbrlock 
U.  S.  Arnty- 


Tuesday,    September  20,  1977 


SESSION  NUMBER  3 


Chairman:    Mr.  Douglas  A.  Morlock 


High  Reliability  Design  Techniques 
for  Distributed  Digital  Systems 

Dr.  M.  P.  Lukas 

Manager  of  the  Systems  Analysis  Dept. 
and 

Mr.  J.  J.  Steinkirchner 
Group  Leader  for  Reliability 
Electronic  Engineering  Dept. 

Bailey  Meter  Cctipany 
29801  Euclid  Avenue 
Wickliffe,  Ohio  44092 


Coffee  Break 


09:00  -  09:50 


09:50  -  10:00 
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10:50  -  11:20     Potentials  of  Initial  Systems  Design 

Techniques  to  Help  Relieve  Critical  Operating 
Skill  Requirements  in  Reactivated  I'lunitions 
Plants 

Dr.  Malcolm  C.  Beaverstock 
Senior  Research  Engineer 
Corporate  Research  Department 
Foxboro  Conpany 
Foxboro,  Massachusetts  02035 


11:20  -  11:30     Discussion  of  Morning  Reports 


SESSION  NUMBER  4 


Chairman:    Professor  Theodore  J.  Williams 
Purdue  Laboratory  for  Applied 

Industrial  Control 
Schools  of  Engineering 
Purdue  University 
West  Lafayette,  Indiana  47907 


11:30  -  12:00     Discussion  of  a  Set  of  Directed  Questions 
to  Obtain  the  Concliasions  of  the  Workshop 


PM 

12:00  -  01:30  Lunch 


01:30  -  02:30     Further  Discussion  of  the  Set  of  Directed 
Questions 

02:30  -  04:00  Formulation  of  the  Conclusions  of  the  Workshop 
04 : 00  Adj  oumment 
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systems,  and  whole  structures.  The  series  presents  research 
results,  test  methods,  and  performance  criteria  related  to  the 
structural  and  environmental  functions  and  the  durability 
and  safety  characteristics  of  building  elements  and  systems. 
Technical  Notes — Studies  or  reports  which  are  complete  in 
themselves  but  restrictive  in  their  treatment  of  a  subject. 
Analogous  to  monographs  but  not  so  comprehensive  in 
scope  or  definitive  in  treatment  of  the  subject  area.  Often 
serve  as  a  vehicle  for  final  reports  of  work  performed  at 
NBS  under  the  sponsorship  of  other  government  agencies. 
Voluntary  Product  Standards — Developed  under  procedures 
published  by  the  Department  of  Commerce  in  Part  10, 
Title  15,  of  the  Code  of  Federal  Regulations.  The  purpose 
of  the  standards  is  to  establish  nationally  recognized  require- 
ments for  products,  and  to  provide  all  concerned  interests 
with  a  basis  for  common  understanding  of  the  characteristics 
of  the  products.  NBS  administers  this  program  as  a  supple- 
ment to  the  activities  of  the  private  sector  standardizing 
organizations. 

Consumer  Information  Series — Practical  information,  based 
on  NBS  research  and  experience,  covering  areas  of  interest 
to  the  consumer.  Easily  understandable  language  and 
illustrations  provide  useful  background  knowledge  for  shop- 
ping in  today's  technological  marketplace. 
Order  above  NBS  publications  from:  Superintendent  of 
Documents,  Government  Printing  Office,  Washington,  D.C. 
20402. 

Order  following  NBS  publications— NBSIR's  and  FIPS  from 
the  National  Technical  Information  Services,  Springfield, 
Va.  22161. 

Federal  Information  Processing  Standards  Publications 
(FIPS  PUB) — Publications  in  this  series  collectively  consti- 
tute the  Federal  Information  Processing  Standards  Register. 
Register  serves  as  the  official  source  of  information  in  the 
Federal  Government  regarding  standards  issued  by  NBS 
pursuant  to  the  Federal  Property  and  Administrative  Serv- 
ices Act  of  1949  as  amended,  Public  Law  89-306  (79  Stat. 
1127),  and  as  implemented  by  Executive  Order  11717 
(38  FR  12315,  dated  May  11,  1973)  and  Part  6  of  Title  15 
CFR  (Code  of  Federal  Regulations). 

NBS  Interagency  Reports  (NBSIR) — A  special  series  of 
interim  or  final  reports  on  work  performed  by  NBS  for 
outside  sponsors  (both  government  and  non-govemment). 
In  general,  initial  distribution  is  handled  by  the  sponsor; 
public  distribution  is  by  the  National  Technical  Information 
Services  (Springfield,  Va.  22161)  in  paper  copy  or  microfiche 
form. 


BIBLIOGRAPHIC  SUBSCRIPTION  SERVICES 


e  following  current-awareness  and  literature-survey  bibli- 
jographies  are  issued  periodically  by  the  Bureau: 
jCryogenic  Data  Center  Current  Awareness  Service.  A  litera- 
ture survey  issued  biweekly.  Annual  subscription:  Domes- 
tic, $25.00;  Foreign,  $30.00. 
Liquified  Natural  Gas.  A  literature  survey  issued  quarterly. 
(Annual  subscription:  $20.00. 


Superconducting  Devices  and  Materials.  A  literature  survey 

issued  quarterly.  Annual  subscription:  $30.00.  Send  subscrip- 
tion orders  and  remittances  for  the  preceding  bibliographic 
services  to  National  Bureau  of  Standards,  Cryogenic  Data 
Center  (275.02)  Boulder,  Colorado  80302. 
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